Showing 1–50 of 71 results for author: Mirzaei, A

CytoBERT: A Foundation Model for Cytometry Data

Authors: Syed Abdul Haseeb Qadri, Bjarne C. Hiller, Felix Blanke, Vanja Sophie Cangalovic, Kutalmış Coşkun, Amin Mirzaei, Tom Siegl, Sebastian Bader, Thomas Kirste, Martin Becker

Abstract: Cytometry measures the complex characteristics of single cells (e.g., counts and protein expression of immune cells) and is widely used across immunological research and clinical settings. However, cytometry data is highly heterogeneous and unstandardized due to experimental protocols and the choice of measured features. While machine learning methods hold the potential to gain deeper insights int… ▽ More Cytometry measures the complex characteristics of single cells (e.g., counts and protein expression of immune cells) and is widely used across immunological research and clinical settings. However, cytometry data is highly heterogeneous and unstandardized due to experimental protocols and the choice of measured features. While machine learning methods hold the potential to gain deeper insights into cell biology, these challenges make them difficult to apply and transfer across studies. Recent advances in foundation models can alleviate these issues, but corresponding approaches are still scarce in this field. To address this, we provide CytoBERT, a publicly available, open-source, open-weight foundation model for single-cell cytometry data with variable marker panels. CytoBERT is pretrained in a self-supervised manner on a large-scale cytometry corpus (15 human datasets with heterogeneous marker panels and more than 50 million cells) curated through marker standardization, enabling it to learn transferable inter-marker relationships within cells. Fine-tuning CytoBERT for sample-level classification demonstrates that transfer learning across heterogeneous cytometry datasets is feasible, providing a starting point for scalable, generalizable cytometry analysis. Code is available at GitHub. △ Less

Submitted 14 August, 2026; originally announced August 2026.

EgoPlay: Event-Triggered Video Editing for Egocentric Streams

Authors: Jinjie Mai, Gordon Guocheng Qian, Willi Menapace, Arpit Sahni, Chaoyang Wang, Ashkan Mirzaei, Runjia Li, Sergey Tulyakov, Bernard Ghanem, Peter Wonka, Rameen Abdal

Abstract: We introduce EgoPlay, an event-triggered video-to-video editor for egocentric streams, obtained by fine-tuning a pretrained V2V diffusion transformer on event-conditioned data built primarily from Ego4D. Given a monocular video and an event-triggered prompt of the form "when X happens, do Y," EgoPlay infers whether and when event X occurs, preserves pre-event frames, and applies edit Y only to the… ▽ More We introduce EgoPlay, an event-triggered video-to-video editor for egocentric streams, obtained by fine-tuning a pretrained V2V diffusion transformer on event-conditioned data built primarily from Ego4D. Given a monocular video and an event-triggered prompt of the form "when X happens, do Y," EgoPlay infers whether and when event X occurs, preserves pre-event frames, and applies edit Y only to the post-event continuation. Rather than cascading a separate event detector with an editor, EgoPlay learns event recognition, temporal restraint, and pixel-level editing jointly in a single end-to-end model, while also handling negative and multi-event prompts. To support this, we construct a large-scale dataset of 106K event-triggered clip-prompt pairs spanning positive triggers, fabricated-trigger negatives, and multi-event prompts. We then train a bidirectional video diffusion editor with event-triggered supervision and derive a causal variant for chunk-by-chunk streamable inference. We further introduce an event-aware evaluation protocol that separately measures post-trigger editing quality, pre-trigger preservation, and false-trigger robustness. On the Ego4D benchmark, EgoPlay substantially outperforms EgoEdit, the state-of-the-art instruction-based egocentric video editing baseline, with relative gains of 17.7%, 16.9%, and 16.4% in editing quality, visual quality, and background consistency. It also surpasses a VLM-guided detector-editor baseline by 15.7%, 14.5%, and 13.5% on the same metrics, while using less than half the GPU memory. △ Less

Submitted 27 July, 2026; originally announced July 2026.

Comments: Accepted to SIGGRAPH Asia 2026 as a Conference Paper. Project page: https://egoplay2026.github.io/egoplay

GeoStream: Toward Precise Camera Controlled Streaming Video Generation

Authors: Yizhou Zhao, Yifan Wang, Xiaoyuan Wang, Yushu Wu, Hao Zhang, Moayed Haji-Ali, Rameen Abdal, Ashkan Mirzaei, Yanyu Li, Willi Menapace, Laszlo Jeni, Sergey Tulyakov, Peter Wonka, Chaoyang Wang

Abstract: Accurate interactive camera control is essential for video-based world models, but most existing approaches learn camera motion implicitly, leading to inaccurate control under out-of-distribution trajectories. Explicit geometric conditioning improves controllability, but existing methods are non-autoregressive and rely on a static 3D cache built from an initial frame, which becomes ineffective onc… ▽ More Accurate interactive camera control is essential for video-based world models, but most existing approaches learn camera motion implicitly, leading to inaccurate control under out-of-distribution trajectories. Explicit geometric conditioning improves controllability, but existing methods are non-autoregressive and rely on a static 3D cache built from an initial frame, which becomes ineffective once the viewpoint moves beyond the original frustum. We propose GeoStream, a framework that enables precise metric-scale camera control in autoregressive streaming video generation. Our method maintains a self-refreshing 3D cache that is periodically updated online from the model's own outputs: we estimate depth from the most recently generated frame, unproject to 3D, and reproject into the target view to produce point reprojections as geometric conditioning for subsequent synthesis. By the same principle, the conditioning seen during training is also rendered from the student's own generated frames, yielding a fully on-policy distillation that naturally aligns the train and inference conditioning distributions. Unlike prior work that uses off-policy condition noising, our approach trains the model against the exact error distribution it encounters at inference, mitigating both standard autoregressive drift and the second-order geometric feedback loop that arises when the cache itself is derived from generated outputs. Quantitative and qualitative results show that our approach substantially improves camera controllability. △ Less

Submitted 13 June, 2026; originally announced June 2026.

Cosmos 3: Omnimodal World Models for Physical AI

Authors: NVIDIA, :, Aditi, Niket Agarwal, Arslan Ali, Jon Allen, Martin Antolini, Adeline Aubame, Alisson Azzolini, Junjie Bai, Maciej Bala, Yogesh Balaji, Josh Bapst, Aarti Basant, Mukesh Beladiya, Mohammad Qazim Bhat, Zaid Pervaiz Bhat, Dan Blick, Vanni Brighella, Han Cai, Tiffany Cai, Eric Cameracci, Jiaxin Cao, Yulong Cao, Mark Carlson , et al. (271 additional authors not shown)

Abstract: We introduce Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences within a unified mixture-of-transformers architecture. By supporting highly flexible input-output configurations, Cosmos 3 seamlessly unifies critical modalities for Physical AI -- effectively subsuming vision-language models, video generators, worl… ▽ More We introduce Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences within a unified mixture-of-transformers architecture. By supporting highly flexible input-output configurations, Cosmos 3 seamlessly unifies critical modalities for Physical AI -- effectively subsuming vision-language models, video generators, world simulators, and world-action models into a single framework. Our evaluation demonstrates that Cosmos 3 establishes a new state-of-the-art across a diverse suite of understanding and generation tasks, demonstrating omnimodal world models as scalable, general-purpose backbones for embodied agents. Our post-trained Cosmos 3 models were ranked as the best open-source Text-to-Image and Image-to-Video models by Artificial Analysis, and the best policy model by RoboArena at the time the technical report was written. To accelerate open research and deployment in Physical AI, we make our code, model checkpoints, curated synthetic datasets, and evaluation benchmark available under the Linux Foundation's OpenMDW-1.1 License at https://github.com/nvidia/cosmos and https://huggingface.co/collections/nvidia/cosmos3. The project website is available at https://research.nvidia.com/labs/cosmos-lab/cosmos3. △ Less

Submitted 23 June, 2026; v1 submitted 1 June, 2026; originally announced June 2026.

Grain Growth Kinetics in (Cr,Mo,Ta,V,W)C1-δ High-Entropy Carbide Ceramics

Authors: Ali Sarikhani, Gregory E. Hilmas, David W. Lipke, Douglas E. Wolfe, Stefano Curtarolo, Shen J. Dillon, Ahmad Mirzaei, William G. Fahrenholtz

Abstract: Understanding grain-boundary mobility during spark plasma sintering can enable microstructure control in high-entropy carbides, yet quantitative grain-growth kinetics remain scarce. In this work, grain growth kinetics and densification behavior were investigated for single-phase fully dense (Cr,Mo,Ta,V,W)C1-δ high-entropy carbide ceramics. Specimens were densified by spark plasma sintering for a c… ▽ More Understanding grain-boundary mobility during spark plasma sintering can enable microstructure control in high-entropy carbides, yet quantitative grain-growth kinetics remain scarce. In this work, grain growth kinetics and densification behavior were investigated for single-phase fully dense (Cr,Mo,Ta,V,W)C1-δ high-entropy carbide ceramics. Specimens were densified by spark plasma sintering for a constant dwell time of 10 min at temperatures between 1750 °C and 1950 °C to isolate the role of temperature on microstructural evolution. Increasing sintering temperature produced grain growth and increased lattice parameter, while maintaining a single-phase rock salt structure. Elemental mapping showed a progressive reduction of Ta segregation with increasing sintering temperature, suggesting enhanced chemical homogenization at elevated temperatures. Grain growth kinetics were analyzed using a normal grain growth model with an assumed growth exponent of n=3, physically reasonable for grain-boundary-controlled growth influenced by solute and vacancy pinning. Arrhenius analysis of the growth factor yielded an apparent activation energy of approximately 620 kJ mol-1, comparable to diffusion-controlled processes in refractory transition-metal carbides. Densification curves revealed rapid consolidation prior to reaching the peak temperature followed by temperature-dominated grain coarsening. These results establish quantitative relationships between densification temperature, grain growth, and diffusion kinetics in a carbide system, providing insight into the microstructural stability of high-entropy, ultra-high-temperature carbide ceramics. △ Less

Submitted 10 April, 2026; originally announced April 2026.

Concurrent Permissive Strategy Templates

Authors: Ashwani Anand, Christel Baier, Calvin Chau, Sascha Klüppelholz, Ali Mirzaei, Satya Prakash Nayak, Anne-Kathrin Schmuck

Abstract: Two-player games on finite graphs provide a rigorous foundation for modeling the strategic interaction between reactive systems and their environment. While concurrent game semantics naturally capture the synchronous interactions characteristic of many cyber-physical systems (CPS), their adoption in CPS design remains limited. Building on the concept of permissive strategy templates (PeSTels) for… ▽ More Two-player games on finite graphs provide a rigorous foundation for modeling the strategic interaction between reactive systems and their environment. While concurrent game semantics naturally capture the synchronous interactions characteristic of many cyber-physical systems (CPS), their adoption in CPS design remains limited. Building on the concept of permissive strategy templates (PeSTels) for turn-based games, we introduce concurrent (permissive) strategy templates (ConSTels) -- a novel representation for sets of randomized winning strategies in concurrent games with Safety, Büchi, and Co-Büchi objectives. ConSTels compactly encode infinite families of strategies, thereby supporting both offline and online adaptation. Offline, we exploit compositionality to enable incremental synthesis: combining ConSTels for simpler objectives into non-conflicting templates for more complex combined objectives. Online, we demonstrate how ConSTels facilitate runtime adaptation, adjusting action probabilities in response to observed opponent behavior to optimize performance while preserving correctness. We implemented ConSTel synthesis and adaptation in a prototype tool and experimentally show its potential. △ Less

Submitted 19 January, 2026; originally announced January 2026.

Comments: TACAS 2026

AI-Native Integrated Sensing and Communications for Self-Organizing Wireless Networks: Architectures, Learning Paradigms, and System-Level Design

Authors: S. Zhang, M. Feizarefi, A. F. Mirzaei

Abstract: Integrated Sensing and Communications (ISAC) is emerging as a foundational paradigm for next-generation wireless networks, enabling communication infrastructures to simultaneously support data transmission and environment sensing. By tightly coupling radio sensing with communication functions, ISAC unlocks new capabilities for situational awareness, localization, tracking, and network adaptation.… ▽ More Integrated Sensing and Communications (ISAC) is emerging as a foundational paradigm for next-generation wireless networks, enabling communication infrastructures to simultaneously support data transmission and environment sensing. By tightly coupling radio sensing with communication functions, ISAC unlocks new capabilities for situational awareness, localization, tracking, and network adaptation. At the same time, the increasing scale, heterogeneity, and dynamics of future wireless systems demand self-organizing network intelligence capable of autonomously managing resources, topology, and services. Artificial intelligence (AI), particularly learning-driven and data-centric methods, has become a key enabler for realizing this vision. This survey provides a comprehensive and system-level review of AI-native ISAC-enabled self-organizing wireless networks. We develop a unified taxonomy that spans: (i) ISAC signal models and sensing modalities, (ii) network state abstraction and perception from sensing-aware radio data, (iii) learning-driven self-organization mechanisms for resource allocation, topology control, and mobility management, and (iv) cross-layer architectures integrating sensing, communication, and network intelligence. We further examine emerging learning paradigms, including deep reinforcement learning, graph-based learning, multi-agent coordination, and federated intelligence that enable autonomous adaptation under uncertainty, mobility, and partial observability. Practical considerations such as sensing-communication trade-offs, scalability, latency, reliability, and security are discussed alongside representative evaluation methodologies and performance metrics. Finally, we identify key open challenges and future research directions toward deployable, trustworthy, and scalable AI-native ISAC systems for 6G and beyond. △ Less

Submitted 29 December, 2025; originally announced January 2026.

EasyV2V: A High-quality Instruction-based Video Editing Framework

Authors: Jinjie Mai, Chaoyang Wang, Guocheng Gordon Qian, Willi Menapace, Sergey Tulyakov, Bernard Ghanem, Peter Wonka, Ashkan Mirzaei

Abstract: While image editing has advanced rapidly, video editing remains less explored, facing challenges in consistency, control, and generalization. We study the design space of data, architecture, and control, and introduce \emph{EasyV2V}, a simple and effective framework for instruction-based video editing. On the data side, we compose existing experts with fast inverses to build diverse video pairs, l… ▽ More While image editing has advanced rapidly, video editing remains less explored, facing challenges in consistency, control, and generalization. We study the design space of data, architecture, and control, and introduce \emph{EasyV2V}, a simple and effective framework for instruction-based video editing. On the data side, we compose existing experts with fast inverses to build diverse video pairs, lift image edit pairs into videos via single-frame supervision and pseudo pairs with shared affine motion, mine dense-captioned clips for video pairs, and add transition supervision to teach how edits unfold. On the model side, we observe that pretrained text-to-video models possess editing capability, motivating a simplified design. Simple sequence concatenation for conditioning with light LoRA fine-tuning suffices to train a strong model. For control, we unify spatiotemporal control via a single mask mechanism and support optional reference images. Overall, EasyV2V works with flexible inputs, e.g., video+text, video+mask+text, video+mask+reference+text, and achieves state-of-the-art video editing results, surpassing concurrent and commercial systems. Project page: https://snap-research.github.io/easyv2v/ △ Less

Submitted 18 December, 2025; originally announced December 2025.

Comments: Project page: https://snap-research.github.io/easyv2v/

OmniView: An All-Seeing Diffusion Model for 3D and 4D View Synthesis

Authors: Xiang Fan, Sharath Girish, Vivek Ramanujan, Chaoyang Wang, Ashkan Mirzaei, Petr Sushko, Aliaksandr Siarohin, Sergey Tulyakov, Ranjay Krishna

Abstract: Prior approaches injecting camera control into diffusion models have focused on specific subsets of 4D consistency tasks: novel view synthesis, text-to-video with camera control, image-to-video, amongst others. Therefore, these fragmented approaches are trained on disjoint slices of available 3D/4D data. We introduce OmniView, a unified framework that generalizes across a wide range of 4D consiste… ▽ More Prior approaches injecting camera control into diffusion models have focused on specific subsets of 4D consistency tasks: novel view synthesis, text-to-video with camera control, image-to-video, amongst others. Therefore, these fragmented approaches are trained on disjoint slices of available 3D/4D data. We introduce OmniView, a unified framework that generalizes across a wide range of 4D consistency tasks. Our method separately represents space, time, and view conditions, enabling flexible combinations of these inputs. For example, OmniView can synthesize novel views from static, dynamic, and multiview inputs, extrapolate trajectories forward and backward in time, and create videos from text or image prompts with full camera control. OmniView is competitive with task-specific models across diverse benchmarks and metrics, improving image quality scores among camera-conditioned diffusion models by up to 33\% in multiview NVS LLFF dataset, 60\% in dynamic NVS Neural 3D Video benchmark, 20\% in static camera control on RE-10K, and reducing camera trajectory errors by 4x in text-conditioned video generation. With strong generalizability in one model, OmniView demonstrates the feasibility of a generalist 4D video model. Project page is available at https://snap-research.github.io/OmniView/ △ Less

Submitted 21 January, 2026; v1 submitted 11 December, 2025; originally announced December 2025.

Comments: Project page: https://snap-research.github.io/OmniView/

EgoEdit: Dataset, Real-Time Streaming Model, and Benchmark for Egocentric Video Editing

Authors: Runjia Li, Moayed Haji-Ali, Ashkan Mirzaei, Chaoyang Wang, Arpit Sahni, Ivan Skorokhodov, Aliaksandr Siarohin, Tomas Jakab, Junlin Han, Sergey Tulyakov, Philip Torr, Willi Menapace

Abstract: We study instruction-guided editing of egocentric videos for interactive AR applications. While recent AI video editors perform well on third-person footage, egocentric views present unique challenges - including rapid egomotion and frequent hand-object interactions - that create a significant domain gap. Moreover, existing offline editing pipelines suffer from high latency, limiting real-time int… ▽ More We study instruction-guided editing of egocentric videos for interactive AR applications. While recent AI video editors perform well on third-person footage, egocentric views present unique challenges - including rapid egomotion and frequent hand-object interactions - that create a significant domain gap. Moreover, existing offline editing pipelines suffer from high latency, limiting real-time interaction. To address these issues, we present a complete ecosystem for egocentric video editing. First, we construct EgoEditData, a carefully designed and manually curated dataset specifically designed for egocentric editing scenarios, featuring rich hand-object interactions, while explicitly preserving hands. Second, we develop EgoEdit, an instruction-following egocentric video editor that supports real-time streaming inference on a single GPU. Finally, we introduce EgoEditBench, an evaluation suite targeting instruction faithfulness, hand and interaction preservation, and temporal stability under egomotion. Across both egocentric and general editing tasks, EgoEdit produces temporally stable, instruction-faithful results with interactive latency. It achieves clear gains on egocentric editing benchmarks-where existing methods struggle-while maintaining performance comparable to the strongest baselines on general editing tasks. EgoEditData and EgoEditBench will be made public for the research community. See our website at https://snap-research.github.io/EgoEdit △ Less

Submitted 5 December, 2025; originally announced December 2025.

Comments: Project page: https://snap-research.github.io/EgoEdit

LoRAQuant: Mixed-Precision Quantization of LoRA to Ultra-Low Bits

Authors: Amir Reza Mirzaei, Yuqiao Wen, Yanshuai Cao, Lili Mou

Abstract: Low-Rank Adaptation (LoRA) has become a popular technique for parameter-efficient fine-tuning of large language models (LLMs). In many real-world scenarios, multiple adapters are loaded simultaneously to enable LLM customization for personalized user experiences or to support a diverse range of tasks. Although each adapter is lightweight in isolation, their aggregate cost becomes substantial at sc… ▽ More Low-Rank Adaptation (LoRA) has become a popular technique for parameter-efficient fine-tuning of large language models (LLMs). In many real-world scenarios, multiple adapters are loaded simultaneously to enable LLM customization for personalized user experiences or to support a diverse range of tasks. Although each adapter is lightweight in isolation, their aggregate cost becomes substantial at scale. To address this, we propose LoRAQuant, a mixed-precision post-training quantization method tailored to LoRA. Specifically, LoRAQuant reparameterizes each adapter by singular value decomposition (SVD) to concentrate the most important information into specific rows and columns. This makes it possible to quantize the important components to higher precision, while quantizing the rest to ultra-low bitwidth. We conduct comprehensive experiments with LLaMA 2-7B, LLaMA 2-13B, and Mistral 7B models on mathematical reasoning, coding, and summarization tasks. Results show that our LoRAQuant uses significantly lower bits than other quantization methods, but achieves comparable or even higher performance. △ Less

Submitted 12 August, 2026; v1 submitted 30 October, 2025; originally announced October 2025.

Track, Inpaint, Resplat: Subject-driven 3D and 4D Generation with Progressive Texture Infilling

Authors: Shuhong Zheng, Ashkan Mirzaei, Igor Gilitschenski

Abstract: Current 3D/4D generation methods are usually optimized for photorealism, efficiency, and aesthetics. However, they often fail to preserve the semantic identity of the subject across different viewpoints. Adapting generation methods with one or few images of a specific subject (also known as Personalization or Subject-driven generation) allows generating visual content that align with the identity… ▽ More Current 3D/4D generation methods are usually optimized for photorealism, efficiency, and aesthetics. However, they often fail to preserve the semantic identity of the subject across different viewpoints. Adapting generation methods with one or few images of a specific subject (also known as Personalization or Subject-driven generation) allows generating visual content that align with the identity of the subject. However, personalized 3D/4D generation is still largely underexplored. In this work, we introduce TIRE (Track, Inpaint, REsplat), a novel method for subject-driven 3D/4D generation. It takes an initial 3D asset produced by an existing 3D generative model as input and uses video tracking to identify the regions that need to be modified. Then, we adopt a subject-driven 2D inpainting model for progressively infilling the identified regions. Finally, we resplat the modified 2D multi-view observations back to 3D while still maintaining consistency. Extensive experiments demonstrate that our approach significantly improves identity preservation in 3D/4D generation compared to state-of-the-art methods. Our project website is available at https://zsh2000.github.io/track-inpaint-resplat.github.io/. △ Less

Submitted 27 October, 2025; originally announced October 2025.

Comments: NeurIPS 2025, 38 pages, 22 figures

Towards In-Situ Failure Assessment: Deep Learning on DIC Results for Laminated Composites

Authors: Amir Mohammad Mirzaei

Abstract: Predicting fracture load in laminated composites with stress raisers is challenging due to complex failure mechanisms such as delamination, fibre breakage, and matrix cracking, which are heavily influenced by fibre orientation, layup sequence, and notch geometry. This study aims to address this by developing a novel deep learning framework that leverages solely experimental strain field data from… ▽ More Predicting fracture load in laminated composites with stress raisers is challenging due to complex failure mechanisms such as delamination, fibre breakage, and matrix cracking, which are heavily influenced by fibre orientation, layup sequence, and notch geometry. This study aims to address this by developing a novel deep learning framework that leverages solely experimental strain field data from Digital Image Correlation (DIC) for accurate, in-situ predictions--bypassing the need for finite element simulations or empirical calibrations. Two complementary architectures are explored: a multi-layer perceptron (MLP) that processes numerical values of maximum principal strain from a targeted rectangular region ahead of the notch, enhanced by advanced feature selection (mutual information, Lasso, and SHAP) to focus on critical data points; and a convolutional neural network (CNN) trained on full-field strain images, bolstered by data augmentation to handle variability and prevent overfitting. Validated across 116 quasi-static tests encompassing 31 distinct configurations--including six layups (quasi-isotropic to highly anisotropic) with four off-axis angles for open-hole specimens, and one cross-ply layup with four off-axis and four on-axis notch orientations for U-notched specimens--the MLP and CNN achieve coefficients of determination (R^2) of 0.86 and 0.82, respectively. This framework captures a broad spectrum of damage modes and responses, from brittle fibre-dominated fracture to ductile delamination-driven failure, and due to its computational efficiency and reliance only on DIC measurements, the approach enables practical in-situ fracture load estimation. △ Less

Submitted 17 October, 2025; originally announced October 2025.

ShapeGen4D: Towards High Quality 4D Shape Generation from Videos

Authors: Jiraphon Yenphraphai, Ashkan Mirzaei, Jianqi Chen, Jiaxu Zou, Sergey Tulyakov, Raymond A. Yeh, Peter Wonka, Chaoyang Wang

Abstract: Video-conditioned 4D shape generation aims to recover time-varying 3D geometry and view-consistent appearance directly from an input video. In this work, we introduce a native video-to-4D shape generation framework that synthesizes a single dynamic 3D representation end-to-end from the video. Our framework introduces three key components based on large-scale pre-trained 3D models: (i) a temporal a… ▽ More Video-conditioned 4D shape generation aims to recover time-varying 3D geometry and view-consistent appearance directly from an input video. In this work, we introduce a native video-to-4D shape generation framework that synthesizes a single dynamic 3D representation end-to-end from the video. Our framework introduces three key components based on large-scale pre-trained 3D models: (i) a temporal attention that conditions generation on all frames while producing a time-indexed dynamic representation; (ii) a time-aware point sampling and 4D latent anchoring that promote temporally consistent geometry and texture; and (iii) noise sharing across frames to enhance temporal stability. Our method accurately captures non-rigid motion, volume changes, and even topological transitions without per-frame optimization. Across diverse in-the-wild videos, our method improves robustness and perceptual fidelity and reduces failure modes compared with the baselines. △ Less

Submitted 7 October, 2025; originally announced October 2025.

Comments: Project page: https://shapegen4d.github.io/

CORE-3D: Context-aware Open-vocabulary Retrieval by Embeddings in 3D

Authors: Mohamad Amin Mirzaei, Pantea Amoie, Ali Ekhterachian, Matin Mirzababaei, Babak Khalaj

Abstract: Object retrieval from a scene has become a new trend of research due to its numerous applications. Recent approaches achieve zero-shot, open-vocabulary 3D semantic mapping by assigning embedding vectors to 2D class-agnostic masks generated via vision-language models (VLMs) and projecting these into 3D. However, these methods often produce fragmented masks and inaccurate semantic assignments due to… ▽ More Object retrieval from a scene has become a new trend of research due to its numerous applications. Recent approaches achieve zero-shot, open-vocabulary 3D semantic mapping by assigning embedding vectors to 2D class-agnostic masks generated via vision-language models (VLMs) and projecting these into 3D. However, these methods often produce fragmented masks and inaccurate semantic assignments due to the direct use of raw masks, limiting their effectiveness in complex environments. To address this, we leverage SemanticSAM with progressive granularity refinement to generate more accurate and numerous object-level masks, mitigating the over-segmentation commonly observed in mask generation models such as vanilla SAM, and improving downstream 3D semantic segmentation. To further enhance semantic context, we employ a context-aware CLIP encoding strategy that integrates multiple contextual views of each mask using empirically determined weighting, providing much richer visual context. We evaluate our approach on multiple 3D scene understanding tasks, including 3D semantic segmentation and object retrieval from language queries, across several benchmark datasets. Experimental results demonstrate significant improvements over existing methods, highlighting the effectiveness of our approach. △ Less

Submitted 12 August, 2026; v1 submitted 29 September, 2025; originally announced September 2025.

AI Safety Assurance in Electric Vehicles: A Case Study on AI-Driven SOC Estimation

Authors: Martin Skoglund, Fredrik Warg, Aria Mirzai, Anders Thorsen, Karl Lundgren, Peter Folkesson, Bastian Havers-zulka

Abstract: Integrating Artificial Intelligence (AI) technology in electric vehicles (EV) introduces unique challenges for safety assurance, particularly within the framework of ISO 26262, which governs functional safety in the automotive domain. Traditional assessment methodologies are not geared toward evaluating AI-based functions and require evolving standards and practices. This paper explores how an ind… ▽ More Integrating Artificial Intelligence (AI) technology in electric vehicles (EV) introduces unique challenges for safety assurance, particularly within the framework of ISO 26262, which governs functional safety in the automotive domain. Traditional assessment methodologies are not geared toward evaluating AI-based functions and require evolving standards and practices. This paper explores how an independent assessment of an AI component in an EV can be achieved when combining ISO 26262 with the recently released ISO/PAS 8800, whose scope is AI safety for road vehicles. The AI-driven State of Charge (SOC) battery estimation exemplifies the process. Key features relevant to the independent assessment of this extended evaluation approach are identified. As part of the evaluation, robustness testing of the AI component is conducted using fault injection experiments, wherein perturbed sensor inputs are systematically introduced to assess the component's resilience to input variance. △ Less

Submitted 3 September, 2025; originally announced September 2025.

Comments: 12 pages, 9 figures, EVS38, https://evs38-program.org/en/evs-38-proceedings/all

Forgetting: A New Mechanism Towards Better Large Language Model Fine-tuning

Authors: Ali Taheri, Alireza Taban, Qizhou Wang, Shanshan Ye, Abdolreza Mirzaei, Tongliang Liu, Bo Han

Abstract: Supervised fine-tuning (SFT) plays a critical role for pretrained large language models (LLMs), notably enhancing their capacity to acquire domain-specific knowledge while preserving or potentially augmenting their general-purpose capabilities. However, the efficacy of SFT hinges on data quality as well as data volume, otherwise it may result in limited performance gains or even degradation relati… ▽ More Supervised fine-tuning (SFT) plays a critical role for pretrained large language models (LLMs), notably enhancing their capacity to acquire domain-specific knowledge while preserving or potentially augmenting their general-purpose capabilities. However, the efficacy of SFT hinges on data quality as well as data volume, otherwise it may result in limited performance gains or even degradation relative to the associated baselines. To mitigate such reliance, we suggest categorizing tokens within each corpus into two parts -- positive and negative tokens -- based on whether they are useful to improve model performance. Positive tokens can be trained in common ways, whereas negative tokens, which may lack essential semantics or be misleading, should be explicitly forgotten. Overall, the token categorization facilitates the model to learn less informative messages, and the forgetting guides the model on what information to learn more precisely. We conduct experiments across diverse and well-established benchmarks using various model architectures, demonstrating that this forgetting mechanism enhances model performance. △ Less

Submitted 28 March, 2026; v1 submitted 6 August, 2025; originally announced August 2025.

Journal ref: Transactions on Machine Learning Research (TMLR), 03/2026

DELTAv2: Accelerating Dense 3D Tracking

Authors: Tuan Duc Ngo, Ashkan Mirzaei, Guocheng Qian, Hanwen Liang, Chuang Gan, Evangelos Kalogerakis, Peter Wonka, Chaoyang Wang

Abstract: We propose a novel algorithm for accelerating dense long-term 3D point tracking in videos. Through analysis of existing state-of-the-art methods, we identify two major computational bottlenecks. First, transformer-based iterative tracking becomes expensive when handling a large number of trajectories. To address this, we introduce a coarse-to-fine strategy that begins tracking with a small subset… ▽ More We propose a novel algorithm for accelerating dense long-term 3D point tracking in videos. Through analysis of existing state-of-the-art methods, we identify two major computational bottlenecks. First, transformer-based iterative tracking becomes expensive when handling a large number of trajectories. To address this, we introduce a coarse-to-fine strategy that begins tracking with a small subset of points and progressively expands the set of tracked trajectories. The newly added trajectories are initialized using a learnable interpolation module, which is trained end-to-end alongside the tracking network. Second, we propose an optimization that significantly reduces the cost of correlation feature computation, another key bottleneck in prior methods. Together, these improvements lead to a 5-100x speedup over existing approaches while maintaining state-of-the-art tracking accuracy. △ Less

Submitted 9 December, 2025; v1 submitted 1 August, 2025; originally announced August 2025.

Comments: Project page: https://snap-research.github.io/DELTAv2/

4Real-Video-V2: Fused View-Time Attention and Feedforward Reconstruction for 4D Scene Generation

Authors: Chaoyang Wang, Ashkan Mirzaei, Vidit Goel, Willi Menapace, Aliaksandr Siarohin, Avalon Vinella, Michael Vasilkovsky, Ivan Skorokhodov, Vladislav Shakhrai, Sergey Korolev, Sergey Tulyakov, Peter Wonka

Abstract: We propose the first framework capable of computing a 4D spatio-temporal grid of video frames and 3D Gaussian particles for each time step using a feed-forward architecture. Our architecture has two main components, a 4D video model and a 4D reconstruction model. In the first part, we analyze current 4D video diffusion architectures that perform spatial and temporal attention either sequentially o… ▽ More We propose the first framework capable of computing a 4D spatio-temporal grid of video frames and 3D Gaussian particles for each time step using a feed-forward architecture. Our architecture has two main components, a 4D video model and a 4D reconstruction model. In the first part, we analyze current 4D video diffusion architectures that perform spatial and temporal attention either sequentially or in parallel within a two-stream design. We highlight the limitations of existing approaches and introduce a novel fused architecture that performs spatial and temporal attention within a single layer. The key to our method is a sparse attention pattern, where tokens attend to others in the same frame, at the same timestamp, or from the same viewpoint. In the second part, we extend existing 3D reconstruction algorithms by introducing a Gaussian head, a camera token replacement algorithm, and additional dynamic layers and training. Overall, we establish a new state of the art for 4D generation, improving both visual quality and reconstruction capability. △ Less

Submitted 18 June, 2025; originally announced June 2025.

DenseDPO: Fine-Grained Temporal Preference Optimization for Video Diffusion Models

Authors: Ziyi Wu, Anil Kag, Ivan Skorokhodov, Willi Menapace, Ashkan Mirzaei, Igor Gilitschenski, Sergey Tulyakov, Aliaksandr Siarohin

Abstract: Direct Preference Optimization (DPO) has recently been applied as a post-training technique for text-to-video diffusion models. To obtain training data, annotators are asked to provide preferences between two videos generated from independent noise. However, this approach prohibits fine-grained comparisons, and we point out that it biases the annotators towards low-motion clips as they often conta… ▽ More Direct Preference Optimization (DPO) has recently been applied as a post-training technique for text-to-video diffusion models. To obtain training data, annotators are asked to provide preferences between two videos generated from independent noise. However, this approach prohibits fine-grained comparisons, and we point out that it biases the annotators towards low-motion clips as they often contain fewer visual artifacts. In this work, we introduce DenseDPO, a method that addresses these shortcomings by making three contributions. First, we create each video pair for DPO by denoising corrupted copies of a ground truth video. This results in aligned pairs with similar motion structures while differing in local details, effectively neutralizing the motion bias. Second, we leverage the resulting temporal alignment to label preferences on short segments rather than entire clips, yielding a denser and more precise learning signal. With only one-third of the labeled data, DenseDPO greatly improves motion generation over vanilla DPO, while matching it in text alignment, visual quality, and temporal consistency. Finally, we show that DenseDPO unlocks automatic preference annotation using off-the-shelf Vision Language Models (VLMs): GPT accurately predicts segment-level preferences similar to task-specifically fine-tuned video reward models, and DenseDPO trained on these labels achieves performance close to using human labels. △ Less

Submitted 10 October, 2025; v1 submitted 3 June, 2025; originally announced June 2025.

Comments: NeurIPS 2025 Spotlight. Project page: https://snap-research.github.io/DenseDPO/

CMA-ES with Radial Basis Function Surrogate for Black-Box Optimization

Authors: Farshid Farhadi Khouzani, Abdolreza Mirzaei, Paul La Plante, Laxmi Gewali

Abstract: Evolutionary optimization algorithms often face defects and limitations that complicate the evolution processes or even prevent them from reaching the global optimum. A notable constraint pertains to the considerable quantity of function evaluations required to achieve the intended solution. This concern assumes heightened significance when addressing costly optimization problems. However, recent… ▽ More Evolutionary optimization algorithms often face defects and limitations that complicate the evolution processes or even prevent them from reaching the global optimum. A notable constraint pertains to the considerable quantity of function evaluations required to achieve the intended solution. This concern assumes heightened significance when addressing costly optimization problems. However, recent research has shown that integrating machine learning methods, specifically surrogate models, with evolutionary optimization can enhance various aspects of these algorithms. Among the evolutionary algorithms, the Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES) is particularly favored. This preference is due to its use of Gaussian distribution for calculating evolution and its ability to adapt optimization parameters, which reduces the need for user intervention in adjusting initial parameters. In this research endeavor, we propose the adoption of surrogate models within the CMA-ES framework called CMA-SAO to develop an initial surrogate model that facilitates the adaptation of optimization parameters through the acquisition of pertinent information derived from the associated surrogate model. Empirical validation reveals that CMA-SAO algorithm markedly diminishes the number of function evaluations in comparison to prevailing algorithms, thereby providing a significant enhancement in operational efficiency. △ Less

Submitted 21 May, 2025; originally announced May 2025.

Comments: 6 pages, ITNG 2025 Conference

Journal ref: In: Proc. ITNG 2025, AISC 1463, Springer (2025)

Assessing wildfire susceptibility in Iran: Leveraging machine learning for geospatial analysis of climatic and anthropogenic factors

Authors: Ehsan Masoudian, Ali Mirzaei, Hossein Bagheri

Abstract: This study investigates the multifaceted factors influencing wildfire risk in Iran, focusing on the interplay between climatic conditions and human activities. Utilizing advanced remote sensing, geospatial information system (GIS) processing techniques such as cloud computing, and machine learning algorithms, this research analyzed the impact of climatic parameters, topographic features, and human… ▽ More This study investigates the multifaceted factors influencing wildfire risk in Iran, focusing on the interplay between climatic conditions and human activities. Utilizing advanced remote sensing, geospatial information system (GIS) processing techniques such as cloud computing, and machine learning algorithms, this research analyzed the impact of climatic parameters, topographic features, and human-related factors on wildfire susceptibility assessment and prediction in Iran. Multiple scenarios were developed for this purpose based on the data sampling strategy. The findings revealed that climatic elements such as soil moisture, temperature, and humidity significantly contribute to wildfire susceptibility, while human activities-particularly population density and proximity to powerlines-also played a crucial role. Furthermore, the seasonal impact of each parameter was separately assessed during warm and cold seasons. The results indicated that human-related factors, rather than climatic variables, had a more prominent influence during the seasonal analyses. This research provided new insights into wildfire dynamics in Iran by generating high-resolution wildfire susceptibility maps using advanced machine learning classifiers. The generated maps identified high risk areas, particularly in the central Zagros region, the northeastern Hyrcanian Forest, and the northern Arasbaran forest, highlighting the urgent need for effective fire management strategies. △ Less

Submitted 20 May, 2025; originally announced May 2025.

Journal ref: Trees, Forests and People, Volume 19, March 2025, 100774

SubROC: AUC-Based Discovery of Exceptional Subgroup Performance for Binary Classifiers

Authors: Tom Siegl, Kutalmış Coşkun, Bjarne C. Hiller, Amin Mirzaei, Florian Lemmerich, Martin Becker

Abstract: Machine learning (ML) is increasingly employed in real-world applications like medicine or economics, thus, potentially affecting large populations. However, ML models often do not perform homogeneously, leading to underperformance or, conversely, unusually high performance in certain subgroups (e.g., sex=female AND marital_status=married). Identifying such subgroups can support practical decision… ▽ More Machine learning (ML) is increasingly employed in real-world applications like medicine or economics, thus, potentially affecting large populations. However, ML models often do not perform homogeneously, leading to underperformance or, conversely, unusually high performance in certain subgroups (e.g., sex=female AND marital_status=married). Identifying such subgroups can support practical decisions on which subpopulation a model is safe to deploy or where more training data is required. However, an efficient and coherent framework for effective search is missing. Consequently, we introduce SubROC, an open-source, easy-to-use framework based on Exceptional Model Mining for reliably and efficiently finding strengths and weaknesses of classification models in the form of interpretable population subgroups. SubROC incorporates common evaluation measures (ROC and PR AUC), efficient search space pruning for fast exhaustive subgroup search, control for class imbalance, adjustment for redundant patterns, and significance testing. We illustrate the practical benefits of SubROC in case studies as well as in comparative analyses across multiple datasets. △ Less

Submitted 27 August, 2025; v1 submitted 16 May, 2025; originally announced May 2025.

Comments: 45 pages, 8 figures; clarify based on reviews, unify experiments to all use the same model type

Informed, but Not Always Improved: Challenging the Benefit of Background Knowledge in GNNs

Authors: Kutalmış Coşkun, Ivo Kavisanczki, Amin Mirzaei, Tom Siegl, Bjarne C. Hiller, Stefan Lüdtke, Martin Becker

Abstract: In complex and low-data domains such as biomedical research, incorporating background knowledge (BK) graphs, such as protein-protein interaction (PPI) networks, into graph-based machine learning pipelines is a promising research direction. However, while BK is often assumed to improve model performance, its actual contribution and the impact of imperfect knowledge remain poorly understood. In this… ▽ More In complex and low-data domains such as biomedical research, incorporating background knowledge (BK) graphs, such as protein-protein interaction (PPI) networks, into graph-based machine learning pipelines is a promising research direction. However, while BK is often assumed to improve model performance, its actual contribution and the impact of imperfect knowledge remain poorly understood. In this work, we investigate the role of BK in an important real-world task: cancer subtype classification. Surprisingly, we find that (i) state-of-the-art GNNs using BK perform no better than uninformed models like linear regression, and (ii) their performance remains largely unchanged even when the BK graph is heavily perturbed. To understand these unexpected results, we introduce an evaluation framework, which employs (i) a synthetic setting where the BK is clearly informative and (ii) a set of perturbations that simulate various imperfections in BK graphs. With this, we test the robustness of BK-aware models in both synthetic and real-world biomedical settings. Our findings reveal that careful alignment of GNN architectures and BK characteristics is necessary but holds the potential for significant performance improvements. △ Less

Submitted 22 September, 2025; v1 submitted 16 May, 2025; originally announced May 2025.

Comments: 10 pages, 7 figures, added repo link

Security Analysis of Chain-FS service

Authors: Vanessa Teague, Arash Mirzaei

Abstract: We examine the security of a cloud storage service that makes very strong claims about the ``trustless'' nature of its security. We find that, although stored files are end-to-end encrypted, the encryption method allows for effective dictionary attacks by a malicious server when passwords only just meet the minimum length required. Furthermore, the file sharing function simply sends the decryption… ▽ More We examine the security of a cloud storage service that makes very strong claims about the ``trustless'' nature of its security. We find that, although stored files are end-to-end encrypted, the encryption method allows for effective dictionary attacks by a malicious server when passwords only just meet the minimum length required. Furthermore, the file sharing function simply sends the decryption passwords to the server with no protection other than TLS. △ Less

Submitted 30 March, 2025; originally announced March 2025.

GraPLUS: Graph-based Placement Using Semantics for Image Composition

Authors: Mir Mohammad Khaleghi, Mehran Safayani, Abdolreza Mirzaei

Abstract: We present GraPLUS (Graph-based Placement Using Semantics), a novel framework for plausible object placement in images that leverages scene graphs and large language models. Our approach uniquely combines graph-structured scene representation with semantic understanding to determine contextually appropriate object positions. The framework employs GPT-2 to transform categorical node and edge labels… ▽ More We present GraPLUS (Graph-based Placement Using Semantics), a novel framework for plausible object placement in images that leverages scene graphs and large language models. Our approach uniquely combines graph-structured scene representation with semantic understanding to determine contextually appropriate object positions. The framework employs GPT-2 to transform categorical node and edge labels into rich semantic embeddings that capture both definitional characteristics and typical spatial contexts, enabling nuanced understanding of object relationships and placement patterns. GraPLUS achieves placement accuracy of 92.1% and an FID score of 28.83 on the OPA dataset, outperforming state-of-the-art methods by 8.1% while maintaining competitive visual quality. In human evaluation studies involving 964 samples assessed by 19 participants, our method was preferred in 52.1% of cases, significantly outperforming previous approaches. The framework's key innovations include: (i) leveraging pre-trained scene graph models that transfer knowledge from other domains, (ii) edge-aware graph neural networks that process scene semantics through structured relationships, (iii) a cross-modal attention mechanism that aligns categorical embeddings with enhanced scene features, and (iv) a multiobjective training strategy incorporating semantic consistency constraints. △ Less

Submitted 19 March, 2025; originally announced March 2025.

Comments: 17 pages, 3 figures, 6 tables

Stress, Strain, or Displacement? A Novel Machine Learning Based Framework to Predict Mixed Mode I/II Fracture Toughness

Authors: Amir Mohammad Mirzaei

Abstract: Accurate prediction of fracture toughness under complex loading conditions, like mixed mode I/II, is essential for reliable failure assessment. This paper aims to develop a machine learning framework for predicting fracture toughness and crack initiation angles by directly utilizing stress, strain, or displacement distributions represented by selected nodes as input features. Validation is conduct… ▽ More Accurate prediction of fracture toughness under complex loading conditions, like mixed mode I/II, is essential for reliable failure assessment. This paper aims to develop a machine learning framework for predicting fracture toughness and crack initiation angles by directly utilizing stress, strain, or displacement distributions represented by selected nodes as input features. Validation is conducted using experimental data across various mode mixities and specimen geometries for brittle materials. Among stress, strain, and displacement fields, it is shown that the stress-based features, when paired with Multilayer Perceptron models, achieve high predictive accuracy with R2 scores exceeding 0.86 for fracture load predictions and 0.94 for angle predictions. A comparison with the Theory of Critical Distances (Generalized Maximum Tangential Stress) demonstrates the high accuracy of the framework. Furthermore, the impact of input parameter selections is studied, and it is demonstrated that advanced feature selection algorithms enable the framework to handle different ranges and densities of the representing field. The framework's performance was further validated for datasets with a limited number of data points and restricted mode mixities, where it maintained high accuracy. The proposed framework is computationally efficient and practical, and it operates without any supplementary post-processing steps, such as stress intensity factor calculations. △ Less

Submitted 1 March, 2025; originally announced March 2025.

MSC Class: 74R10; 74G70

An analysis of the combination of feature selection and machine learning methods for an accurate and timely detection of lung cancer

Authors: Omid Shahriyar, Babak Nuri Moghaddam, Davoud Yousefi, Abbas Mirzaei, Farnaz Hoseini

Abstract: One of the deadliest cancers, lung cancer necessitates an early and precise diagnosis. Because patients have a better chance of recovering, early identification of lung cancer is crucial. This review looks at how to diagnose lung cancer using sophisticated machine learning techniques like Random Forest (RF) and Support Vector Machine (SVM). The Chi-squared test is one feature selection strategy th… ▽ More One of the deadliest cancers, lung cancer necessitates an early and precise diagnosis. Because patients have a better chance of recovering, early identification of lung cancer is crucial. This review looks at how to diagnose lung cancer using sophisticated machine learning techniques like Random Forest (RF) and Support Vector Machine (SVM). The Chi-squared test is one feature selection strategy that has been successfully applied to find related features and enhance model performance. The findings demonstrate that these techniques can improve detection efficiency and accuracy while also assisting in runtime reduction. This study produces recommendations for further research as well as ideas to enhance diagnostic techniques. In order to improve healthcare and create automated methods for detecting lung cancer, this research is a critical first step. △ Less

Submitted 19 January, 2025; originally announced January 2025.

3DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splatting

Authors: Qi Wu, Janick Martinez Esturo, Ashkan Mirzaei, Nicolas Moenne-Loccoz, Zan Gojcic

Abstract: 3D Gaussian Splatting (3DGS) enables efficient reconstruction and high-fidelity real-time rendering of complex scenes on consumer hardware. However, due to its rasterization-based formulation, 3DGS is constrained to ideal pinhole cameras and lacks support for secondary lighting effects. Recent methods address these limitations by tracing the particles instead, but, this comes at the cost of signif… ▽ More 3D Gaussian Splatting (3DGS) enables efficient reconstruction and high-fidelity real-time rendering of complex scenes on consumer hardware. However, due to its rasterization-based formulation, 3DGS is constrained to ideal pinhole cameras and lacks support for secondary lighting effects. Recent methods address these limitations by tracing the particles instead, but, this comes at the cost of significantly slower rendering. In this work, we propose 3D Gaussian Unscented Transform (3DGUT), replacing the EWA splatting formulation with the Unscented Transform that approximates the particles through sigma points, which can be projected exactly under any nonlinear projection function. This modification enables trivial support of distorted cameras with time dependent effects such as rolling shutter, while retaining the efficiency of rasterization. Additionally, we align our rendering formulation with that of tracing-based methods, enabling secondary ray tracing required to represent phenomena such as reflections and refraction within the same 3D representation. The source code is available at: https://github.com/nv-tlabs/3dgrut. △ Less

Submitted 24 March, 2025; v1 submitted 16 December, 2024; originally announced December 2024.

Comments: Our paper has been accepted by CVPR 2025. For more details and updates, please visit our project website: https://research.nvidia.com/labs/toronto-ai/3DGUT

EventSplat: 3D Gaussian Splatting from Moving Event Cameras for Real-time Rendering

Authors: Toshiya Yura, Ashkan Mirzaei, Igor Gilitschenski

Abstract: We introduce a method for using event camera data in novel view synthesis via Gaussian Splatting. Event cameras offer exceptional temporal resolution and a high dynamic range. Leveraging these capabilities allows us to effectively address the novel view synthesis challenge in the presence of fast camera motion. For initialization of the optimization process, our approach uses prior knowledge encod… ▽ More We introduce a method for using event camera data in novel view synthesis via Gaussian Splatting. Event cameras offer exceptional temporal resolution and a high dynamic range. Leveraging these capabilities allows us to effectively address the novel view synthesis challenge in the presence of fast camera motion. For initialization of the optimization process, our approach uses prior knowledge encoded in an event-to-video model. We also use spline interpolation for obtaining high quality poses along the event camera trajectory. This enhances the reconstruction quality from fast-moving cameras while overcoming the computational limitations traditionally associated with event-based Neural Radiance Field (NeRF) methods. Our experimental evaluation demonstrates that our results achieve higher visual fidelity and better performance than existing event-based NeRF approaches while being an order of magnitude faster to render. △ Less

Submitted 21 May, 2025; v1 submitted 10 December, 2024; originally announced December 2024.

Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videos

Authors: Hanxue Liang, Jiawei Ren, Ashkan Mirzaei, Antonio Torralba, Ziwei Liu, Igor Gilitschenski, Sanja Fidler, Cengiz Oztireli, Huan Ling, Zan Gojcic, Jiahui Huang

Abstract: Recent advancements in static feed-forward scene reconstruction have demonstrated significant progress in high-quality novel view synthesis. However, these models often struggle with generalizability across diverse environments and fail to effectively handle dynamic content. We present BTimer (short for BulletTimer), the first motion-aware feed-forward model for real-time reconstruction and novel… ▽ More Recent advancements in static feed-forward scene reconstruction have demonstrated significant progress in high-quality novel view synthesis. However, these models often struggle with generalizability across diverse environments and fail to effectively handle dynamic content. We present BTimer (short for BulletTimer), the first motion-aware feed-forward model for real-time reconstruction and novel view synthesis of dynamic scenes. Our approach reconstructs the full scene in a 3D Gaussian Splatting representation at a given target ('bullet') timestamp by aggregating information from all the context frames. Such a formulation allows BTimer to gain scalability and generalization by leveraging both static and dynamic scene datasets. Given a casual monocular dynamic video, BTimer reconstructs a bullet-time scene within 150ms while reaching state-of-the-art performance on both static and dynamic scene datasets, even compared with optimization-based approaches. △ Less

Submitted 21 September, 2025; v1 submitted 4 December, 2024; originally announced December 2024.

Comments: Project website: https://research.nvidia.com/labs/toronto-ai/bullet-timer/

GaussianCut: Interactive segmentation via graph cut for 3D Gaussian Splatting

Authors: Umangi Jain, Ashkan Mirzaei, Igor Gilitschenski

Abstract: We introduce GaussianCut, a new method for interactive multiview segmentation of scenes represented as 3D Gaussians. Our approach allows for selecting the objects to be segmented by interacting with a single view. It accepts intuitive user input, such as point clicks, coarse scribbles, or text. Using 3D Gaussian Splatting (3DGS) as the underlying scene representation simplifies the extraction of o… ▽ More We introduce GaussianCut, a new method for interactive multiview segmentation of scenes represented as 3D Gaussians. Our approach allows for selecting the objects to be segmented by interacting with a single view. It accepts intuitive user input, such as point clicks, coarse scribbles, or text. Using 3D Gaussian Splatting (3DGS) as the underlying scene representation simplifies the extraction of objects of interest which are considered to be a subset of the scene's Gaussians. Our key idea is to represent the scene as a graph and use the graph-cut algorithm to minimize an energy function to effectively partition the Gaussians into foreground and background. To achieve this, we construct a graph based on scene Gaussians and devise a segmentation-aligned energy function on the graph to combine user inputs with scene properties. To obtain an initial coarse segmentation, we leverage 2D image/video segmentation models and further refine these coarse estimates using our graph construction. Our empirical evaluations show the adaptability of GaussianCut across a diverse set of scenes. GaussianCut achieves competitive performance with state-of-the-art approaches for 3D segmentation without requiring any additional segmentation-aware training. △ Less

Submitted 12 November, 2024; originally announced November 2024.

MERGE: Matching Electronic Results with Genuine Evidence for verifiable voting in person at remote locations

Authors: Ben Adida, John Caron, Arash Mirzaei, Vanessa Teague

Abstract: Overseas military personnel often face significant challenges in participating in elections due to the slow pace of traditional mail systems, which can result in ballots missing crucial deadlines. While internet-based voting offers a faster alternative, it introduces serious risks to the integrity and privacy of the voting process. We introduce the MERGE protocol to address these issues by combini… ▽ More Overseas military personnel often face significant challenges in participating in elections due to the slow pace of traditional mail systems, which can result in ballots missing crucial deadlines. While internet-based voting offers a faster alternative, it introduces serious risks to the integrity and privacy of the voting process. We introduce the MERGE protocol to address these issues by combining the speed of electronic ballot delivery with the reliability of paper returns. This protocol allows voters to submit an electronic record of their vote quickly while simultaneously mailing a paper ballot for verification. The electronic record can be used for preliminary results, but the paper ballot is used in a Risk Limiting Audit (RLA) if received in time, ensuring the integrity of the election. This approach extends the time window for ballot arrival without undermining the security and accuracy of the vote count. △ Less

Submitted 25 March, 2025; v1 submitted 9 October, 2024; originally announced October 2024.

ComAlign: Compositional Alignment in Vision-Language Models

Authors: Ali Abdollah, Amirmohammad Izadi, Armin Saghafian, Reza Vahidimajd, Mohammad Mozafari, Amirreza Mirzaei, Mohammadmahdi Samiei, Mahdieh Soleymani Baghshah

Abstract: Vision-language models (VLMs) like CLIP have showcased a remarkable ability to extract transferable features for downstream tasks. Nonetheless, the training process of these models is usually based on a coarse-grained contrastive loss between the global embedding of images and texts which may lose the compositional structure of these modalities. Many recent studies have shown VLMs lack composition… ▽ More Vision-language models (VLMs) like CLIP have showcased a remarkable ability to extract transferable features for downstream tasks. Nonetheless, the training process of these models is usually based on a coarse-grained contrastive loss between the global embedding of images and texts which may lose the compositional structure of these modalities. Many recent studies have shown VLMs lack compositional understandings like attribute binding and identifying object relationships. Although some recent methods have tried to achieve finer-level alignments, they either are not based on extracting meaningful components of proper granularity or don't properly utilize the modalities' correspondence (especially in image-text pairs with more ingredients). Addressing these limitations, we introduce Compositional Alignment (ComAlign), a fine-grained approach to discover more exact correspondence of text and image components using only the weak supervision in the form of image-text pairs. Our methodology emphasizes that the compositional structure (including entities and relations) extracted from the text modality must also be retained in the image modality. To enforce correspondence of fine-grained concepts in image and text modalities, we train a lightweight network lying on top of existing visual and language encoders using a small dataset. The network is trained to align nodes and edges of the structure across the modalities. Experimental results on various VLMs and datasets demonstrate significant improvements in retrieval and compositional benchmarks, affirming the effectiveness of our plugin model. △ Less

Submitted 12 September, 2024; originally announced September 2024.

Optimizing RPL Routing Using Tabu Search to Improve Link Stability and Energy Consumption in IoT Networks

Authors: Mehran Tarif, Mohammadhossein Homaei, Abbas Mirzaei, Babak Nouri-Moghaddam

Abstract: In the Internet of Things (IoT) networks, the Routing Protocol for Low-power and Lossy Networks (RPL) is a widely adopted standard due to its efficiency in managing resource-constrained and energy-limited nodes. However, persistent challenges such as high energy consumption, unstable links, and suboptimal routing continue to hinder network performance, affecting both the longevity of the network a… ▽ More In the Internet of Things (IoT) networks, the Routing Protocol for Low-power and Lossy Networks (RPL) is a widely adopted standard due to its efficiency in managing resource-constrained and energy-limited nodes. However, persistent challenges such as high energy consumption, unstable links, and suboptimal routing continue to hinder network performance, affecting both the longevity of the network and the reliability of data transmission. This paper proposes an enhanced RPL routing mechanism by integrating the Tabu Search (TS) optimization algorithm to address these issues. The proposed approach focuses on optimizing the parent and child selection process in the RPL protocol, leveraging a composite cost function that incorporates critical parameters, including Residual Energy, Transmission Energy, Distance to Sink, Hop Count(HC), Expected Transmission Count (ETX), and Link Stability Rate(LSR). Through extensive simulations, we demonstrate that our method significantly improves link stability, reduces energy consumption, and enhances the packet delivery ratio, leading to a more efficient and longer-lasting IoT network. The findings suggest that TS can effectively balance the trade-offs inherent in IoT routing, providing a practical solution for improving the overall performance of RPL-based networks. △ Less

Submitted 16 June, 2025; v1 submitted 13 August, 2024; originally announced August 2024.

Comments: 31 Pages, 22 Figures, 6 Tables

3D Gaussian Ray Tracing: Fast Tracing of Particle Scenes

Authors: Nicolas Moenne-Loccoz, Ashkan Mirzaei, Or Perel, Riccardo de Lutio, Janick Martinez Esturo, Gavriel State, Sanja Fidler, Nicholas Sharp, Zan Gojcic

Abstract: Particle-based representations of radiance fields such as 3D Gaussian Splatting have found great success for reconstructing and re-rendering of complex scenes. Most existing methods render particles via rasterization, projecting them to screen space tiles for processing in a sorted order. This work instead considers ray tracing the particles, building a bounding volume hierarchy and casting a ray… ▽ More Particle-based representations of radiance fields such as 3D Gaussian Splatting have found great success for reconstructing and re-rendering of complex scenes. Most existing methods render particles via rasterization, projecting them to screen space tiles for processing in a sorted order. This work instead considers ray tracing the particles, building a bounding volume hierarchy and casting a ray for each pixel using high-performance GPU ray tracing hardware. To efficiently handle large numbers of semi-transparent particles, we describe a specialized rendering algorithm which encapsulates particles with bounding meshes to leverage fast ray-triangle intersections, and shades batches of intersections in depth-order. The benefits of ray tracing are well-known in computer graphics: processing incoherent rays for secondary lighting effects such as shadows and reflections, rendering from highly-distorted cameras common in robotics, stochastically sampling rays, and more. With our renderer, this flexibility comes at little cost compared to rasterization. Experiments demonstrate the speed and accuracy of our approach, as well as several applications in computer graphics and vision. We further propose related improvements to the basic Gaussian representation, including a simple use of generalized kernel functions which significantly reduces particle hit counts. △ Less

Submitted 9 October, 2024; v1 submitted 9 July, 2024; originally announced July 2024.

Comments: Project page: https://gaussiantracer.github.io/. Published at SIGGRAPH Asia 2024

L4GM: Large 4D Gaussian Reconstruction Model

Authors: Jiawei Ren, Kevin Xie, Ashkan Mirzaei, Hanxue Liang, Xiaohui Zeng, Karsten Kreis, Ziwei Liu, Antonio Torralba, Sanja Fidler, Seung Wook Kim, Huan Ling

Abstract: We present L4GM, the first 4D Large Reconstruction Model that produces animated objects from a single-view video input -- in a single feed-forward pass that takes only a second. Key to our success is a novel dataset of multiview videos containing curated, rendered animated objects from Objaverse. This dataset depicts 44K diverse objects with 110K animations rendered in 48 viewpoints, resulting in… ▽ More We present L4GM, the first 4D Large Reconstruction Model that produces animated objects from a single-view video input -- in a single feed-forward pass that takes only a second. Key to our success is a novel dataset of multiview videos containing curated, rendered animated objects from Objaverse. This dataset depicts 44K diverse objects with 110K animations rendered in 48 viewpoints, resulting in 12M videos with a total of 300M frames. We keep our L4GM simple for scalability and build directly on top of LGM, a pretrained 3D Large Reconstruction Model that outputs 3D Gaussian ellipsoids from multiview image input. L4GM outputs a per-frame 3D Gaussian Splatting representation from video frames sampled at a low fps and then upsamples the representation to a higher fps to achieve temporal smoothness. We add temporal self-attention layers to the base LGM to help it learn consistency across time, and utilize a per-timestep multiview rendering loss to train the model. The representation is upsampled to a higher framerate by training an interpolation model which produces intermediate 3D Gaussian representations. We showcase that L4GM that is only trained on synthetic data generalizes extremely well on in-the-wild videos, producing high quality animated 3D assets. △ Less

Submitted 14 June, 2024; originally announced June 2024.

Comments: Project page: https://research.nvidia.com/labs/toronto-ai/l4gm

Cybersecurity Pathways Towards CE-Certified Autonomous Forestry Machines

Authors: Mazen Mohamad, Ramana Reddy Avula, Peter Folkesson, Pierre Kleberger, Aria Mirzai, Martin Skoglund, Marvin Damschen

Abstract: The increased importance of cybersecurity in autonomous machinery is becoming evident in the forestry domain. Forestry worksites are becoming more complex with the involvement of multiple systems and system of systems. Hence, there is a need to investigate how to address cybersecurity challenges for autonomous systems of systems in the forestry domain. Using a literature review and adapting standa… ▽ More The increased importance of cybersecurity in autonomous machinery is becoming evident in the forestry domain. Forestry worksites are becoming more complex with the involvement of multiple systems and system of systems. Hence, there is a need to investigate how to address cybersecurity challenges for autonomous systems of systems in the forestry domain. Using a literature review and adapting standards from similar domains, as well as collaborative sessions with domain experts, we identify challenges towards CE-certified autonomous forestry machines focusing on cybersecurity and safety. Furthermore, we discuss the relationship between safety and cybersecurity risk assessment and their relation to AI, highlighting the need for a holistic methodology for their assurance. △ Less

Submitted 30 April, 2024; originally announced April 2024.

RefFusion: Reference Adapted Diffusion Models for 3D Scene Inpainting

Authors: Ashkan Mirzaei, Riccardo De Lutio, Seung Wook Kim, David Acuna, Jonathan Kelly, Sanja Fidler, Igor Gilitschenski, Zan Gojcic

Abstract: Neural reconstruction approaches are rapidly emerging as the preferred representation for 3D scenes, but their limited editability is still posing a challenge. In this work, we propose an approach for 3D scene inpainting -- the task of coherently replacing parts of the reconstructed scene with desired content. Scene inpainting is an inherently ill-posed task as there exist many solutions that plau… ▽ More Neural reconstruction approaches are rapidly emerging as the preferred representation for 3D scenes, but their limited editability is still posing a challenge. In this work, we propose an approach for 3D scene inpainting -- the task of coherently replacing parts of the reconstructed scene with desired content. Scene inpainting is an inherently ill-posed task as there exist many solutions that plausibly replace the missing content. A good inpainting method should therefore not only enable high-quality synthesis but also a high degree of control. Based on this observation, we focus on enabling explicit control over the inpainted content and leverage a reference image as an efficient means to achieve this goal. Specifically, we introduce RefFusion, a novel 3D inpainting method based on a multi-scale personalization of an image inpainting diffusion model to the given reference view. The personalization effectively adapts the prior distribution to the target scene, resulting in a lower variance of score distillation objective and hence significantly sharper details. Our framework achieves state-of-the-art results for object removal while maintaining high controllability. We further demonstrate the generality of our formulation on other downstream tasks such as object insertion, scene outpainting, and sparse view reconstruction. △ Less

Submitted 16 April, 2024; originally announced April 2024.

Comments: Project page: https://reffusion.github.io

An adaptive acceleration scheme for phase-field fatigue computations

Authors: Jonas Heinzmann, Pietro Carrara, Marreddy Ambati, Amir Mohammad Mirzaei, Laura De Lorenzis

Abstract: Phase-field models of fatigue are capable of reproducing the main phenomenology of fatigue behavior. However, phase-field computations in the high-cycle fatigue regime are prohibitively expensive, due to the need to resolve spatially the small length scale inherent to phase-field models and temporally the loading history for several millions of cycles. As a remedy, we propose a fully adaptive acce… ▽ More Phase-field models of fatigue are capable of reproducing the main phenomenology of fatigue behavior. However, phase-field computations in the high-cycle fatigue regime are prohibitively expensive, due to the need to resolve spatially the small length scale inherent to phase-field models and temporally the loading history for several millions of cycles. As a remedy, we propose a fully adaptive acceleration scheme based on the cycle jump technique, where the cycle-by-cycle resolution of an appropriately determined number of cycles is skipped while predicting the local system evolution during the jump. The novelty of our approach is a cycle-jump criterion to determine the appropriate cycle-jump size based on a target increment of a global variable which monitors the advancement of fatigue. We propose the definition and meaning of this variable for three general stages of the fatigue life. In comparison to existing acceleration techniques, our approach needs no parameters and bounds for the cycle-jump size, and it works independently of the material, specimen or loading conditions. Since one of the monitoring variables is the fatigue crack length, we introduce an accurate, flexible and efficient method for its computation, which overcomes the issues of conventional crack tip tracking algorithms and enables the consideration of several cracks evolving at the same time. The performance of the proposed acceleration scheme is demonstrated with representative numerical examples, which show a speedup reaching four orders of magnitude in the high-cycle fatigue regime with consistently high accuracy. △ Less

Submitted 14 December, 2024; v1 submitted 10 April, 2024; originally announced April 2024.

Journal ref: Computational Mechanics (2024)

Enhancing crop classification accuracy by synthetic SAR-Optical data generation using deep learning

Authors: Ali Mirzaei, Hossein Bagheri, Iman Khosravi

Abstract: Crop classification using remote sensing data has emerged as a prominent research area in recent decades. Studies have demonstrated that fusing SAR and optical images can significantly enhance the accuracy of classification. However, a major challenge in this field is the limited availability of training data, which adversely affects the performance of classifiers. In agricultural regions, the dom… ▽ More Crop classification using remote sensing data has emerged as a prominent research area in recent decades. Studies have demonstrated that fusing SAR and optical images can significantly enhance the accuracy of classification. However, a major challenge in this field is the limited availability of training data, which adversely affects the performance of classifiers. In agricultural regions, the dominant crops typically consist of one or two specific types, while other crops are scarce. Consequently, when collecting training samples to create a map of agricultural products, there is an abundance of samples from the dominant crops, forming the majority classes. Conversely, samples from other crops are scarce, representing the minority classes. Addressing this issue requires overcoming several challenges and weaknesses associated with traditional data generation methods. These methods have been employed to tackle the imbalanced nature of the training data. Nevertheless, they still face limitations in effectively handling the minority classes. Overall, the issue of inadequate training data, particularly for minority classes, remains a hurdle that traditional methods struggle to overcome. In this research, We explore the effectiveness of conditional tabular generative adversarial network (CTGAN) as a synthetic data generation method based on a deep learning network, in addressing the challenge of limited training data for minority classes in crop classification using the fusion of SAR-optical data. Our findings demonstrate that the proposed method generates synthetic data with higher quality that can significantly increase the number of samples for minority classes leading to better performance of crop classifiers. △ Less

Submitted 3 February, 2024; originally announced February 2024.

Journal ref: ISPRS Int. J. Geo-Inf. 2023, 12(11), 450

Knowledge Distillation on Spatial-Temporal Graph Convolutional Network for Traffic Prediction

Authors: Mohammad Izadi, Mehran Safayani, Abdolreza Mirzaei

Abstract: Efficient real-time traffic prediction is crucial for reducing transportation time. To predict traffic conditions, we employ a spatio-temporal graph neural network (ST-GNN) to model our real-time traffic data as temporal graphs. Despite its capabilities, it often encounters challenges in delivering efficient real-time predictions for real-world traffic data. Recognizing the significance of timely… ▽ More Efficient real-time traffic prediction is crucial for reducing transportation time. To predict traffic conditions, we employ a spatio-temporal graph neural network (ST-GNN) to model our real-time traffic data as temporal graphs. Despite its capabilities, it often encounters challenges in delivering efficient real-time predictions for real-world traffic data. Recognizing the significance of timely prediction due to the dynamic nature of real-time data, we employ knowledge distillation (KD) as a solution to enhance the execution time of ST-GNNs for traffic prediction. In this paper, We introduce a cost function designed to train a network with fewer parameters (the student) using distilled data from a complex network (the teacher) while maintaining its accuracy close to that of the teacher. We use knowledge distillation, incorporating spatial-temporal correlations from the teacher network to enable the student to learn the complex patterns perceived by the teacher. However, a challenge arises in determining the student network architecture rather than considering it inadvertently. To address this challenge, we propose an algorithm that utilizes the cost function to calculate pruning scores, addressing small network architecture search issues, and jointly fine-tunes the network resulting from each pruning stage using KD. Ultimately, we evaluate our proposed ideas on two real-world datasets, PeMSD7 and PeMSD8. The results indicate that our method can maintain the student's accuracy close to that of the teacher, even with the retention of only 3% of network parameters. △ Less

Submitted 24 September, 2024; v1 submitted 22 January, 2024; originally announced January 2024.

Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representations

Authors: Tristan Aumentado-Armstrong, Ashkan Mirzaei, Marcus A. Brubaker, Jonathan Kelly, Alex Levinshtein, Konstantinos G. Derpanis, Igor Gilitschenski

Abstract: Neural Radiance Fields (NeRFs) have proven to be powerful 3D representations, capable of high quality novel view synthesis of complex scenes. While NeRFs have been applied to graphics, vision, and robotics, problems with slow rendering speed and characteristic visual artifacts prevent adoption in many use cases. In this work, we investigate combining an autoencoder (AE) with a NeRF, in which laten… ▽ More Neural Radiance Fields (NeRFs) have proven to be powerful 3D representations, capable of high quality novel view synthesis of complex scenes. While NeRFs have been applied to graphics, vision, and robotics, problems with slow rendering speed and characteristic visual artifacts prevent adoption in many use cases. In this work, we investigate combining an autoencoder (AE) with a NeRF, in which latent features (instead of colours) are rendered and then convolutionally decoded. The resulting latent-space NeRF can produce novel views with higher quality than standard colour-space NeRFs, as the AE can correct certain visual artifacts, while rendering over three times faster. Our work is orthogonal to other techniques for improving NeRF efficiency. Further, we can control the tradeoff between efficiency and image quality by shrinking the AE architecture, achieving over 13 times faster rendering with only a small drop in performance. We hope that our approach can form the basis of an efficient, yet high-fidelity, 3D scene representation for downstream tasks, especially when retaining differentiability is useful, as in many robotics scenarios requiring continual learning. △ Less

Submitted 26 October, 2023; originally announced October 2023.

ACM Class: I.2.10

Stress, Strain, or Energy: Which One Is the Superior Parameter to Estimate Fatigue Life of Notched Components? An Answer by a Novel Machine Learning-Based Framework

Authors: Amir Mohammad Mirzaei

Abstract: This paper introduces a simple framework for accurately predicting the fatigue lifetime of notched components by employing various machine learning algorithms applied to a wide range of materials, loading conditions, notch geometries, and fatigue lives. Traditional approaches for this task have relied on empirical relationships involving one of the mechanical properties, such as stress, strain, or… ▽ More This paper introduces a simple framework for accurately predicting the fatigue lifetime of notched components by employing various machine learning algorithms applied to a wide range of materials, loading conditions, notch geometries, and fatigue lives. Traditional approaches for this task have relied on empirical relationships involving one of the mechanical properties, such as stress, strain, or energy. This study goes further by exploring which mechanical property serves as a better measure. The key idea of the framework is to use the gradient of the mechanical properties (stress, strain, and energy) to distinguish between different notch geometries. To demonstrate the accuracy and broad applicability of the framework, it is initially validated using isotropic materials, subsequently applied to samples produced through additive manufacturing techniques, and ultimately tested on carbon fiber laminated composites. The research demonstrates that the gradient of all three measures can be effectively employed to estimate fatigue lifetime, with stress-based predictions exhibiting the highest accuracy. Among the machine learning algorithms investigated, Gradient Boosting and Random Forest yield the most successful results. A noteworthy finding is the significant improvement in prediction accuracy achieved by incorporating new data generated based on the Basquin equation. △ Less

Submitted 18 October, 2023; originally announced October 2023.

An overview of VANET vehicular networks

Authors: Ali Hozouri, Abbas Mirzaei, Shiva RazaghZadeh, Davoud Yousefi

Abstract: Today, with the development of intercity and metropolitan roadways and with various cars moving in various directions, there is a greater need than ever for a network to coordinate commutes. Nowadays, people spend a lot of time in their vehicles. Smart automobiles have developed to make that time safer, more effective, more fun, pollution-free, and affordable. However, maintaining the optimum use… ▽ More Today, with the development of intercity and metropolitan roadways and with various cars moving in various directions, there is a greater need than ever for a network to coordinate commutes. Nowadays, people spend a lot of time in their vehicles. Smart automobiles have developed to make that time safer, more effective, more fun, pollution-free, and affordable. However, maintaining the optimum use of resources and addressing rising needs continues to be a challenge given the popularity of vehicle users and the growing diversity of requests for various services. As a result, VANET will require modernized working practices in the future. Modern intelligent transportation management and driver assistance systems are created using cutting-edge communication technology. Vehicular Ad-hoc networks promise to increase transportation effectiveness, accident prevention, and pedestrian comfort by allowing automobiles and road infrastructure to communicate entertainment and traffic information. By constructing thorough frameworks, workflow patterns, and update procedures, including block-chain, artificial intelligence, and SDN (Software Defined Networking), this paper addresses VANET-related technologies, future advances, and related challenges. An overview of the VANET upgrade solution is given in this document in order to handle potential future problems. △ Less

Submitted 12 September, 2023; originally announced September 2023.

Watch Your Steps: Local Image and Scene Editing by Text Instructions

Authors: Ashkan Mirzaei, Tristan Aumentado-Armstrong, Marcus A. Brubaker, Jonathan Kelly, Alex Levinshtein, Konstantinos G. Derpanis, Igor Gilitschenski

Abstract: Denoising diffusion models have enabled high-quality image generation and editing. We present a method to localize the desired edit region implicit in a text instruction. We leverage InstructPix2Pix (IP2P) and identify the discrepancy between IP2P predictions with and without the instruction. This discrepancy is referred to as the relevance map. The relevance map conveys the importance of changing… ▽ More Denoising diffusion models have enabled high-quality image generation and editing. We present a method to localize the desired edit region implicit in a text instruction. We leverage InstructPix2Pix (IP2P) and identify the discrepancy between IP2P predictions with and without the instruction. This discrepancy is referred to as the relevance map. The relevance map conveys the importance of changing each pixel to achieve the edits, and is used to to guide the modifications. This guidance ensures that the irrelevant pixels remain unchanged. Relevance maps are further used to enhance the quality of text-guided editing of 3D scenes in the form of neural radiance fields. A field is trained on relevance maps of training views, denoted as the relevance field, defining the 3D region within which modifications should be made. We perform iterative updates on the training views guided by rendered relevance maps from the relevance field. Our method achieves state-of-the-art performance on both image and NeRF editing tasks. Project page: https://ashmrz.github.io/WatchYourSteps/ △ Less

Submitted 17 August, 2023; originally announced August 2023.

Comments: Project page: https://ashmrz.github.io/WatchYourSteps/

Journal ref: European Conference on Computer Vision (ECCV) 2024

A Novel Approach for Establishing Connectivity in Partitioned Mobile Sensor Networks Using Beamforming Techniques

Authors: Abbas Mirzaei, Shahram Zandiyan

Abstract: Network connectivity is one of the major design issues in the context of mobile sensor networks. Due to diverse communication patterns, some nodes lying in high-traffic zones may consume more energy and eventually die out resulting in network partitioning. This phenomenon may deprive a large number of alive nodes of sending their important time critical data to the sink. The application of data ca… ▽ More Network connectivity is one of the major design issues in the context of mobile sensor networks. Due to diverse communication patterns, some nodes lying in high-traffic zones may consume more energy and eventually die out resulting in network partitioning. This phenomenon may deprive a large number of alive nodes of sending their important time critical data to the sink. The application of data caching in mobile sensor networks is exponentially increasing as a high-speed data storage layer. This paper presents a deep learning-based beamforming approach to find the optimal transmission strategies for cache-enabled backhaul networks. In the proposed scheme, the sensor nodes in isolated partitions work together to form a directional beam which significantly increases their overall communication range to reach out a distant relay node connected to the main part of the network. The proposed methodology of cooperative beamforming-based partition connectivity works efficiently if an isolated cluster gets partitioned with a favorably large number of nodes. We also present a new cross-layer method for link cost that makes a balance between the energy used by the relay. By directly adding the accessible auxiliary nodes to the set of routing links, the algorithm chooses paths which provide maximum dynamic beamforming usage for the intermediate nodes. The proposed approach is then evaluated through simulation results. The simulation results show that the proposed mechanism achieves up to 30% energy consumption reduction through beamforming as partition healing in addition to guarantee user throughput. △ Less

Submitted 9 August, 2023; originally announced August 2023.

Fast Classification with Sequential Feature Selection in Test Phase

Authors: Ali Mirzaei, Vahid Pourahmadi, Hamid Sheikhzadeh, Alireza Abdollahpourrostam

Abstract: This paper introduces a novel approach to active feature acquisition for classification, which is the task of sequentially selecting the most informative subset of features to achieve optimal prediction performance during testing while minimizing cost. The proposed approach involves a new lazy model that is significantly faster and more efficient compared to existing methods, while still producing… ▽ More This paper introduces a novel approach to active feature acquisition for classification, which is the task of sequentially selecting the most informative subset of features to achieve optimal prediction performance during testing while minimizing cost. The proposed approach involves a new lazy model that is significantly faster and more efficient compared to existing methods, while still producing comparable accuracy results. During the test phase, the proposed approach utilizes Fisher scores for feature ranking to identify the most important feature at each step. In the next step the training dataset is filtered based on the observed value of the selected feature and then we continue this process to reach to acceptable accuracy or limit of the budget for feature acquisition. The performance of the proposed approach was evaluated on synthetic and real datasets, including our new synthetic dataset, CUBE dataset and also real dataset Forest. The experimental results demonstrate that our approach achieves competitive accuracy results compared to existing methods, while significantly outperforming them in terms of speed. The source code of the algorithm is released at github with this link: https://github.com/alimirzaei/FCwSFS. △ Less

Submitted 25 June, 2023; originally announced June 2023.

Reference-guided Controllable Inpainting of Neural Radiance Fields

Authors: Ashkan Mirzaei, Tristan Aumentado-Armstrong, Marcus A. Brubaker, Jonathan Kelly, Alex Levinshtein, Konstantinos G. Derpanis, Igor Gilitschenski

Abstract: The popularity of Neural Radiance Fields (NeRFs) for view synthesis has led to a desire for NeRF editing tools. Here, we focus on inpainting regions in a view-consistent and controllable manner. In addition to the typical NeRF inputs and masks delineating the unwanted region in each view, we require only a single inpainted view of the scene, i.e., a reference view. We use monocular depth estimator… ▽ More The popularity of Neural Radiance Fields (NeRFs) for view synthesis has led to a desire for NeRF editing tools. Here, we focus on inpainting regions in a view-consistent and controllable manner. In addition to the typical NeRF inputs and masks delineating the unwanted region in each view, we require only a single inpainted view of the scene, i.e., a reference view. We use monocular depth estimators to back-project the inpainted view to the correct 3D positions. Then, via a novel rendering technique, a bilateral solver can construct view-dependent effects in non-reference views, making the inpainted region appear consistent from any view. For non-reference disoccluded regions, which cannot be supervised by the single reference view, we devise a method based on image inpainters to guide both the geometry and appearance. Our approach shows superior performance to NeRF inpainting baselines, with the additional advantage that a user can control the generated scene via a single inpainted image. Project page: https://ashmrz.github.io/reference-guided-3d △ Less

Submitted 20 April, 2023; v1 submitted 19 April, 2023; originally announced April 2023.

Comments: Project Page: https://ashmrz.github.io/reference-guided-3d

CAMM: Building Category-Agnostic and Animatable 3D Models from Monocular Videos

Authors: Tianshu Kuai, Akash Karthikeyan, Yash Kant, Ashkan Mirzaei, Igor Gilitschenski

Abstract: Animating an object in 3D often requires an articulated structure, e.g. a kinematic chain or skeleton of the manipulated object with proper skinning weights, to obtain smooth movements and surface deformations. However, existing models that allow direct pose manipulations are either limited to specific object categories or built with specialized equipment. To reduce the work needed for creating an… ▽ More Animating an object in 3D often requires an articulated structure, e.g. a kinematic chain or skeleton of the manipulated object with proper skinning weights, to obtain smooth movements and surface deformations. However, existing models that allow direct pose manipulations are either limited to specific object categories or built with specialized equipment. To reduce the work needed for creating animatable 3D models, we propose a novel reconstruction method that learns an animatable kinematic chain for any articulated object. Our method operates on monocular videos without prior knowledge of the object's shape or underlying structure. Our approach is on par with state-of-the-art 3D surface reconstruction methods on various articulated object categories while enabling direct pose manipulations by re-posing the learned kinematic chain. △ Less

Submitted 14 April, 2023; originally announced April 2023.

Comments: Project Page: https://camm3d.github.io/