Athens AI, Machine Learning and Computer Vision Meetup

Jul 08, 2026
9 AM - 11 AM PT
Online. Register for Zoom!

Some Modalities Are More Equal Than Others: Understanding and Improving Multimodal Integration in MLLMs

Multimodal large language models can process vision, audio, and text, but it remains unclear whether they truly integrate these modalities or rely on shortcut cues. In this talk, I will present our recent work, “Some Modalities Are More Equal Than Others,” where we introduce MMA-Bench, a benchmark designed to probe MLLMs under controlled audio–visual conflict, misleading text, and modality-specific queries. Through black-box evaluation and white-box attention analysis, we show that current MLLMs often struggle when modalities disagree, exhibit model-specific modality biases, and can be distracted by irrelevant textual context. We further propose an alignment-aware tuning strategy that trains models to answer based on the queried modality, improving robustness and multimodal grounding. This talk will highlight both the failure modes of current MLLMs and practical directions toward more reliable cross-modal reasoning.

About the Speaker

Tianle Chen is a Ph.D. student in Computer Science at Boston University, advised by Prof. Deepti Ghadiyaram. His research focuses on multimodal large language models, audio–visual reasoning, robustness, and trustworthy multimodal AI. He is interested in understanding how models allocate evidence across modalities and designing methods that improve reliable multimodal reasoning.

LinkedOut: Linking World Knowledge Out of Video LLMs for Next-Generation Video Recommendation

This CVPR 2026 work links structured world knowledge representations out of Video LLMs for next-generation video recommendation, covering how large vision-language models can provide rich semantic priors for video understanding while addressing efficiency and deployment challenges in real recommendation systems.

About the Speaker

Haichao Zhang is a Ph.D. candidate in Computer Engineering at Northeastern University. His research focuses on computer vision, vision-language models, video understanding and generation, and efficient multimodal foundation models. He has research experience at Google CoreML, Meta Reality Labs, LinkedIn Video AI, Amazon AWS AI Labs, and Tencent.

CylinderDepth: Cylindrical Spatial Attention for Multi-View Consistent Self-Supervised Surround Depth Estimation

This paper presents CylinderDepth, a self-supervised surround depth estimation method leveraging cylindrical spatial attention for multi-view consistency across camera rigs.

About the Speaker

Samer Abualhanud is a PhD student and research staff member at Leibniz University Hannover, Germany, supervised by Dr.-Ing. Max Mehltretter and Prof. Christian Heipke. Research focuses on multi-view consistency in 3D reconstruction.

Your ViT is Secretly Also a Video Segmentation Model

Existing online video segmentation models typically combine a per-frame segmentation module with complex, specialized tracking modules. This work shows that a plain Vision Transformer encoder with a lightweight temporal module can match that performance, resulting in VidEoMT — up to 5–10x faster, running at up to 160 FPS with a ViT-L encoder.

About the Speaker

Daan de Geus is an Assistant Professor in the Mobile Perception Systems Lab at TU/e. He received his PhD (cum laude) from TU/e in 2024, and his research focuses on machine learning for visual and multimodal scene understanding.

Jul 09, 2026
9 AM - 11 AM PT
Online. Register for Zoom!

Efficient Representation and Coding of Dynamic Light Fields

This talk presents a data-driven approach that integrates aperture and pixel-wise exposure coding with Dynamic Mode Decomposition (DMD) to achieve compact representation of dynamic light fields. By modeling them as mathematical dynamical systems, the framework captures coherent structures across all dimensions and achieves scalable compression, bitrate savings, and high-quality reconstructions.

About the Speaker

Joshitha Ravishanker is a PhD scholar in the Department of Electrical Engineering at IIT Madras, supervised by Dr. Mansi Sharma and Dr. Kaushik Mitra. She is a Prime Minister's Research Fellow and her doctoral research focuses on the efficient representation and compression of light fields for display applications.

PHANTOM: Physics-Infused Video Generation via Joint Modeling of Visual and Latent Physical Dynamics

Recent video generation models can produce visually striking results, but they often fail to capture the physical dynamics that govern how real-world scenes evolve. In this talk, I will present PHANTOM, a physics-infused video generation model that jointly predicts visual content and latent physical dynamics. PHANTOM uses a physics-aware video representation to guide generation toward videos that are both visually realistic and physically consistent, without requiring explicit simulator-based physical specifications. I will discuss the model design, key results on standard and physics-aware video generation benchmarks, and how this work supports broader progress toward multimodal world models for physical AI and embodied reasoning.

About the Speaker

Ismini Lourentzou is an Assistant Professor at the University of Illinois Urbana-Champaign and Director of the Perception and LANguage Lab. Her research focuses on multimodal machine learning, vision-language models, generative modeling, and embodied AI, with applications in physical reasoning, robotics, healthcare, and trustworthy AI.

LoST: Level of Semantics Tokenization for 3D Shapes

Tokenization is fundamental to generative modeling and especially important for autoregressive 3D generation. However, current 3D shape tokenizers rely on geometric level-of-detail hierarchies that are token-inefficient and poorly aligned with semantic structure. We propose Level-of-Semantics Tokenization (LoST), which orders tokens by semantic salience so early tokens produce complete, plausible shapes and later tokens refine detailed geometry and semantics.

LoST is trained with Relational Inter-Distance Alignment (RIDA), a semantic alignment loss that matches relationships in 3D shape latent space to those in DINO feature space. Experiments show that LoST achieves state-of-the-art reconstruction and efficient high-quality AR 3D generation while using only 0.1%–10% of the tokens required by prior methods.

About the Speaker

Niladri Dutt is an ELLIS PhD student at University College London (UCL), sponsored by Adobe Research. He is advised by Prof Niloy Mitra (UCL) and Duygu Ceylan (Adobe). His research interests are in representation learning for 3D and multimodal learning.

3D Reconstruction Improves Weakly-Supervised Semantic Segmentation

Semantic segmentation typically requires expensive, dense annotations, making large-scale training a significant bottleneck. We address this by introducing a framework that leverages recent advances in feed-forward 3D reconstruction to improve weakly supervised semantic segmentation on 2D images, using only sparse labels such as points, scribbles, or coarse masks.

Our core insight is that 3D geometric structure recovered directly from casual 2D video sequences provides powerful cross-view consistency constraints that can propagate sparse annotations across entire scenes. A dual student-teacher architecture enforces semantic consistency between 2D images and reconstructed 3D point clouds, injecting geometric supervision into the learning process while keeping inference purely 2D. Our solution achieves state-of-the-art performance, outperforming existing methods by 2–7% across a range of datasets and annotation types, without requiring additional labels or inference overhead.

About the Speaker

Wolfgang Boettcher is an ELLIS doctoral researcher in the Computer Vision and Machine Learning group at the Max Planck Institute for Informatics. Since his master's degree at ETH Zurich, his research focuses on visual perception, semantic scene understanding, and dynamic 3D reconstruction. He is particularly interested in models that can reason about the physical environment for applications in autonomous systems and robotics.

Jul 10, 2026
9 AM - 11 AM PT
Online. Register for Zoom!

Advancing Generative Quality and Reasoning in Multimodal AI

This talk exposes hidden limitations of frontier multimodal models across reasoning and visual generation, demonstrates the inherent brittleness of VLMs and audio-visual MLLMs, and introduces simple yet effective techniques to build robustness. It also covers human-centric metrics for perceptually accurate evaluation of generative media.

About the Speaker

Deepti Ghadiyaram is an Assistant Professor of Computer Science at Boston University. Her research focuses on building safe, interpretable, and robust computer vision systems with advanced reasoning capabilities. Before joining BU she was at Runway and Meta AI, and earned her PhD from UT Austin in 2017.

HyperRealm: Hyperbolic Vision Language Models for Real-World Hierarchical Multimodal Understanding

Real-world multimodal data naturally exhibits hierarchical structure, yet standard VLMs like CLIP align images and text in Euclidean space, which cannot preserve tree-like hierarchies. HyperRealm embeds images and text in a Poincaré ball to encode hierarchical relationships, introducing an adaptive entropy-driven entailment loss. Evaluated on 18 zero-shot classification benchmarks, it shows consistent improvements over Euclidean CLIP baselines.

About the Speaker

Kathy Wu holds a Ph.D. in Applied Mathematics from USC. She is currently an Applied Scientist at Amazon within the Global Store organization, leading projects in e-commerce recommendation, multimodal VLMs, and LLM/GenAI applications. Her research has been published at ICCV, CVPR, ICLR, SIGIR, and WACV.

Cross-Modal Domain Adaptation using Semantic Parametric Mapping

XD-MAP is a framework that transfers semantic knowledge from image datasets to LiDAR by constructing semantic parametric maps from monocular detections and geometric priors. Unlike previous approaches, XD-MAP does not require overlapping sensor views and enables scalable 360° supervision for LiDAR perception without manual annotation.

About the Speaker

Frank Bieder is a researcher in computer vision and autonomous systems, leading the Visual and Spatial Learning group at FZI Research Center for Information Technology. His research covers multimodal perception, map-based learning, and cross-sensor domain adaptation for autonomous driving. He received his Ph.D. from KIT in 2026.

WalkGPT: Pixel-Grounded Navigation Guidance for Pedestrians

Pedestrian navigation requires more than generic scene description; users need to understand walkable areas, obstacles, and the distance of surrounding objects. In this talk, I will present WalkGPT, a grounded vision-language model for accessibility-aware pedestrian navigation. WalkGPT connects language reasoning with segmentation masks and object-level distance estimates to generate grounded navigation guidance from pedestrian-view images. I will also introduce PAVE, a 41k-sample benchmark for depth-aware accessibility reasoning in real pedestrian environments. The talk will highlight how grounded multimodal AI can support safer and more interpretable pedestrian assistance.

About the Speaker

Rafi Ibn Sultan is a Ph.D. researcher in Computer Science at Wayne State University, working on computer vision, multimodal AI, and vision-language models. His research focuses on grounded and interpretable AI systems for real-world visual reasoning, including pedestrian navigation and medical image segmentation. His recent work includes WalkGPT, accepted at CVPR 2026, and GeoSAM, accepted at ECAI 2025.

Jul 21, 2026
9:00 AM - 11:00 AM PST
Online. Register for the Zoom!

Outdoor Robot Navigation in the Unstructured World: From Traversability to Physical Scene Understanding

Outdoor robot navigation in the unstructured world requires robots to reason about more than obstacles: they must understand where they can move, what terrain is suitable, and how scene context affects navigation decisions. In sidewalks, campuses, trails, and off-road environments, these decisions depend on geometric structure, terrain conditions, semantic cues, and robot-environment interaction.

In this talk, I will present our recent work on scene understanding for outdoor navigation, including a large-scale multimodal dataset for studying outdoor traversability, approaches for trajectory generation and selection, vision-language reasoning for contextual navigation, and Gaussian-based 3D scene modeling. I will also discuss how physical reasoning can extend scene understanding from visual and geometric perception toward terrain properties and interaction cues.

Together, these works explore how robots can better interpret unstructured outdoor environments and use that understanding for navigation decision-making.

About the Speaker

Jing Liang is a postdoctoral researcher at the Stanford Robotics Center, working on robot navigation, perception, and human-centered autonomy in complex real-world environments.

Scene Graphs and the Future of Mapping

In this talk, I will question whether 3D reconstruction is still a necessary part of mapping in the age of feedforward models and present some alternatives. Then, I discuss scene graphs as an alternative map representation and their applications for mobile manipulation.

About the Speaker

Hermann Blum is a Junior Professor at the University of Bonn and the Lamarr Institute. Hermann's research focuses on machine learning for robotic perception and scene understanding, developing models and methods to understand an agent's environment semantically and geometrically.

Toward Zero-Shot 6D Pose Estimation and Tracking of Cluttered Objects on Edge Devices

Robust 6D pose estimation of textured objects under diverse illumination conditions remains a significant challenge, often requiring a trade-off between accurate initial pose estimation and efficient real-time tracking. We present a unified framework explicitly designed for efficient execution on edge devices, which fuses a robust initial estimation module with a fast motion-based tracker.

The key to our approach is a shared, lighting-invariant color-pair feature representation that forms a consistent foundation for both stages. For initial estimation, this representation facilitates robust registration between the live RGB-D view and the object's 3D mesh.

For tracking, the same representation validates temporal correspondences, enabling a lightweight model to reliably regress the object's pose. Experiments on benchmark datasets demonstrate that our integrated approach is both effective and robust, providing competitive pose estimation accuracy while maintaining high-fidelity tracking even through abrupt pose changes.

This is joint work with Xingjian Yang.

About the Speaker

Ashis Banerjee is an Associate Professor of Industrial & Systems Engineering and Mechanical Engineering at the University of Washington, Seattle. Prior to joining UW, he was a Research Scientist at GE Global Research and a Postdoctoral Associate at MIT.

Trustworthy Geometric Perception: Certifiable Optimization and Robust Estimation

Autonomous robots in safety-critical settings require geometric perception that is not merely accurate on average, but provably correct under adversarial conditions. Yet most pipelines rely on local optimization methods that fail silently when poorly initialized.

This talk presents GlobustVP, a certifiably optimal vanishing point estimator that reformulates joint VP localization and line association as a quadratically constrained quadratic program (QCQP) and relaxes it to a tight semidefinite program (SDP), achieving the first globally optimal and outlier-robust solution to this problem. Recognized as a Best Paper Award Candidate at CVPR 2025 (top 0.1%, 14 of 13,008 submissions), GlobustVP demonstrates that certifiable global optimization is both practically feasible and highly competitive.

More broadly, this work is part of a research program toward trustworthy geometric perception: systems that know when they are wrong, and can communicate that to the robots and humans that depend on them.

About the Speaker

Zhenjun Zhao I am a postdoctoral researcher at University of Zaragoza, working with Javier Civera.