Hyderabad AI Machine Learning and Computer Vision Meetup

Online. Register for the Zoom!

Multimodal AI for Efficient Medical Imaging Dataset Curation

We present a multimodal AI pipeline to streamline patient selection and quality assessment for radiology AI development. Our system evaluates patient clinical histories, imaging protocols, and data quality, embedding results into imaging metadata. Using FiftyOne researchers can rapidly filter and assemble high-quality cohorts in minutes instead of weeks, freeing radiologists for clinical work and accelerating AI tool development.

About the Speaker

Brandon Konkel is a Senior Machine Learning engineer at Booz Allen Hamilton with over a decade of experience developing AI solutions for medical imaging.

AI-Powered Heart Ultrasound: From Model Training to Real-Time App Deployment

We have built AI-driven tools to automate the assessment of key heart parameters from point-of-care ultrasound, including Right Atrial Pressure (RAP) and Ejection Fraction (EF). In collaboration with UCSF, we trained deep learning models on a proprietary dataset of over 15,000 labeled ultrasound studies and deployed the full pipeline in a real-time iOS app integrated with the Butterfly probe. A UCSF-led clinical trial has validated the RAP workflow, and we are actively expanding the system to support EF prediction using both A4C and PLAX views.

This talk will present our end-to-end pipeline, from dataset development and model training to mobile deployment—demonstrating how AI can enable real-time heart assessments directly at the point of care.

About the Speaker

Jeffrey Gao is a PhD candidate at Caltech, working at the intersection of machine learning and medical imaging. His research focuses on developing clinically deployable AI systems for ultrasound-based heart assessments, with an emphasis on real-time, edge-based inference and system integration.

Let’s Look Deep at Continuous Patient Monitoring

In hospitals, direct patient observation is limited–nurses spend only 37% of their shift engaged in patient care, and physicians average just 10 visits per hospital stay. LookDeep Health’s AI-driven platform enables continuous and passive monitoring of individual patients, and has been deployed “in the wild” for nearly 3 years. They recently published a study validating this system, titled “Continuous Patient Monitoring with AI”. This talk is a technical dive into said paper, focusing on the intersection of AI and real-world application.

About the Speaker

Paolo Gabriel, PhD is a senior AI engineer at LookDeep Health, where they continue to use computer vision and signal processing to augment patient care in the hospital.

AI in Healthcare: Lessons from Oncology Innovation

About the Speaker

Artificial intelligence is rapidly transforming how we diagnose, treat, and manage health.

Dr. Asba (AT) Tasneem is a healthcare data and innovation leader with over 20 years of experience at the intersection of clinical research, AI, and digital health. She has led large-scale programs in oncology and data strategy, partnering with organizations like the FDA, Duke University, and top pharma companies to drive AI-enabled healthcare solutions.

Online. Register for the Zoom!

MedVAE: Efficient Automated Interpretation of Medical Images with Large-Scale Generalizable Autoencoders

We present MedVAE, a family of six generalizable 2D and 3D variational autoencoders trained on over one million images from 19 open-source medical imaging datasets using a novel two-stage training strategy. MedVAE downsizes high-dimensional medical images into compact latent representations, reducing storage by up to 512× and accelerating downstream tasks by up to 70× while preserving clinically relevant features. We demonstrate across 20 evaluation tasks that these latent representations can replace high-resolution images in computer-aided diagnosis pipelines without compromising performance. MedVAE is open-source with a streamlined finetuning pipeline and inference engine, enabling scalable model development in resource-constrained medical imaging settings.

About the Speakers

Ashwin Kumar is a PhD Candidate in Biomedical Physics at Stanford University, advised by Akshay Chaudhari and Greg Zaharchuk. He focuses on developing deep learning methodologies to advance medical image acquisition and analysis.

Maya Varma is a PhD student in computer science at Stanford University. Her research focuses on the development of artificial intelligence methods for addressing healthcare challenges, with a particular focus on medical imaging applications.

Leveraging Foundation Models for Pathology: Progress and Pitfalls

How do you train ML models on pathology slides that are thousands of times larger than standard images? Foundation models offer a breakthrough approach to these gigapixel-scale challenges. This talk explores how self-supervised foundation models trained on broad histopathology datasets are transforming computational pathology. We’ll examine their progress in handling weakly-supervised learning, managing tissue preparation variations, and enabling rapid prototyping with minimal labeled examples. However, significant challenges remain: increasing computational demands, the potential for bias, and questions about generalizability across diverse populations. This talk will offer a balanced perspective to help separate foundation model hype from genuine clinical value.

About the Speaker

Heather D. Couture is a consultant and founder of Pixel Scientia Labs, where she partners with mission-driven founders and R&D teams to support applications of computer vision for people and planetary health. She has a PhD in Computer Science and has published in top-tier computer vision and medical imaging venues. She hosts the Impact AI Podcast and writes regularly on LinkedIn, for her newsletter Computer Vision Insights, and for a variety of other publications.

LesionLocator: Zero-Shot Universal Tumor Segmentation and Tracking in 3D Whole-Body Imaging

Recent advances in promptable segmentation have transformed medical imaging workflows, yet most existing models are constrained to static 2D or 3D applications. This talk presents LesionLocator, the first end-to-end framework for universal 4D lesion segmentation and tracking using dense spatial prompts. The system enables zero-shot tumor analysis across whole-body 3D scans and multiple timepoints, propagating a single user prompt through longitudinal follow-ups to segment and track lesion progression. Trained on over 23,000 annotated scans and supplemented with a synthetic time-series dataset, LesionLocator achieves human-level performance in segmentation and outperforms state-of-the-art baselines in longitudinal tracking tasks. The presentation also highlights advances in 3D interactive segmentation, including our open-set tool nnInteractive, showing how spatial prompting can scale from user-guided interaction to clinical-grade automation.

About the Speaker

Maximilian Rokussis is a PhD scholar at the German Cancer Research Center (DKFZ), working in the Division of Medical Image Computing under Klaus Maier-Hein. He focuses on 3D multimodal and multi-timepoint segmentation with spatial and text prompts. With several MICCAI challenge wins and first-author publications at CVPR and MICCAI, he co-leads the Helmholtz Medical Foundation Model initiative and develops AI solutions at the interface of research and clinical radiology.

LLMs for Smarter Diagnosis: Unlocking the Future of AI in Healthcare

Large Language Models are rapidly transforming the healthcare landscape. In this talk, I will explore how LLMs like GPT-4 and DeepSeek-R1 are being used to support disease diagnosis, predict chronic conditions, and assist medical professionals without relying on sensitive patient data. Drawing from my published research and real-world applications, I’ll discuss the technical challenges, ethical considerations, and the future potential of integrating LLMs in clinical settings. The talk will offer valuable insights for developers, researchers, and healthcare innovators interested in applying AI responsibly and effectively.

About the Speaker

Gaurav K Gupta graduated from Youngstown State University, Bachelor’s in Computer Science and Mathematics.

Where
Online. Register for the Zoom!

What Foundation Models really need to be capable of for Autonomous Driving – The Drive4C Benchmark

Foundation models hold the potential to generalize the driving task and support language-based interaction in autonomous driving. However, they continue to struggle with specific reasoning tasks essential for robotic navigation. Current benchmarks typically provide only aggregate performance scores, making it difficult to assess the underlying capabilities these models require. Drive4C addresses this gap by introducing a closed-loop benchmark that evaluates semantic, spatial, temporal, and physical understanding—enabling more targeted improvements to advance foundation models for autonomous driving.

About the Speaker

Tin Stribor Sohn is a PhD Student at Porsche AG and Karlsruhe Institute of Technology in the area of Foundation Models for Scenario Understanding and Decision Making in Autonomous Robotics, Tech Lead at Data Driven Engineering for Autonomous Driving, Prior: Master in CS at University of Tuebingen with focus on Computer Vision and Deep Learning and co-founder of a software company for smart EV charging.

Human Motion Prediction – Enhanced Realism via Nonisotropic Gaussian Diffusion

Predicting future human motion is a key challenge in generative AI and computer vision, as generated motions should be realistic and diverse at the same time. This talk presents a novel approach that leverages top-performing latent generative diffusion models with a novel paradigm. Nonisotropic Gaussian diffusion leads to better performance, fewer parameters, and faster training at no additional computational cost. We will also discuss how such benefits can be obtained in other application domains.

About the Speaker

Cecilia Curreli is a Ph.D. student at the Technical University of Munich, specializing in generative models. A member of the AI Competence Center at MCML, she has conducted research in deep learning, computer vision, and quantum physics through international collaborations with the University of Tokyo and the Chinese Academy of Science.

Efficient Few-Shot Adaptation of Open-Set Detection Models

We propose an efficient few-shot adaptation method for the Grounding-DINO open-set object detection model, designed to improve performance on domain-specific specialized datasets like agriculture, where extensive annotation is costly. The method circumvents the challenges of manual text prompt engineering by removing the standard text encoder and instead introduces randomly initialized, trainable text embeddings. These embeddings are optimized directly from a few labeled images, allowing the model to quickly adapt to new domains and object classes with minimal data. This approach demonstrates superior performance over zero-shot methods and competes favorably with other few-shot techniques, offering a promising solution for rapid model specialization.

About the Speaker

Dr. Sudhir Sornapudi is a Senior Data Scientist- II at Corteva Agriscience. He leads the Advanced Vision Intelligence team, driving computer vision innovations internally from cell-to-space with Biotechnology, Crop Health, and Seed Operations.

OpticalNet: An Optical Imaging Dataset and Benchmark Beyond the Diffraction Limit

Optical imaging capable of resolving nanoscale features would revolutionize scientific research and engineering applications across biomedicine, smart manufacturing, and semiconductor quality control. However, due to the physical phenomenon of diffraction, the optical resolution is limited to approximately half the wavelength of light, which impedes the observation of subwavelength objects such as the native state coronavirus, typically smaller than 200 nm. Fortunately, deep learning methods have shown remarkable potential in uncovering underlying patterns within data, promising to overcome the diffraction limit by revealing the mapping pattern between diffraction images and their corresponding ground truth object images.

However, the absence of suitable datasets has hindered progress in this field —— collecting high-quality optical data of subwavelength objects is highly difficult as these objects are inherently invisible under conventional microscopy, making it impossible to perform standard visual calibration and drift correction. Therefore, we provide the first general optical imaging dataset based on the “building block” concept for challenging the diffraction limit. Drawing an analogy to modular construction principles, we construct a comprehensive optical imaging dataset comprising subwavelength fundamental elements, i.e., small square units that can be assembled into larger and more complex objects. We then frame the task as an image-to-image translation task and evaluate various vision methods. Experimental results validate our “building block” concept, demonstrating that models trained on basic square units can effectively generalize to realistic, more complex unseen objects. Most importantly, by highlighting this underexplored AI-for-science area and its potential, we aspire to advance optical science by fostering collaboration with the vision and machine learning communities.

About the Speakers

Wang Benquan is the final-year PhD candidate at Nanyang Technological University, Singapore. His research interests are AI for Science, scientific deep learning, optical metrology and imaging.

Ruyi is a PhD at University of Texas at Austin, working on generative models and reinforcement learning, and their applications.

Online. Register for the Zoom!

OFER : Occluded Face Expression Reconstruction

Reconstructing 3D face models from a single image is an inherently ill-posed problem, which becomes even more challenging in the presence of occlusions where multiple reconstructions can be equally valid. Despite the ubiquity of the problem, very few methods address its multi-hypothesis nature.

In this paper we introduce OFER, a novel approach for single-image 3D face reconstruction that can generate plausible, diverse, and expressive 3D faces by training two diffusion models to generate a shape and expression coefficients of face parametric model, conditioned on the input image. To maintain consistency across diverse expressions, the challenge is to select the best matching shape. To achieve this, we propose a novel ranking mechanism that sorts the outputs of the shape diffusion network based on predicted shape accuracy scores.

Paper: OFER: Occluded Face Expression Reconstruction

About the Speaker

Pratheba Selvaraju has a PhD from the University of Massachusetts, Amherst. Currently researcher at Max Planck Institute – Perceived systems. Research Interest is in 3D reconstruction and modeling problem, geometry processing and generative modeling. Currently also exploring the space of virtual try-ons combining vision and 3D techniques.

SmartHome-Bench: Benchmark for Video Anomaly Detection in Smart Homes Using Multi-Modal LMMs

Video anomaly detection is crucial for ensuring safety and security, yet existing benchmarks overlook the unique context of smart home environments. We introduce SmartHome-Bench, a dataset of 1,203 smart home videos annotated according to a novel taxonomy of seven anomaly categories, such as Wildlife, Senior Care, and Baby Monitoring. We evaluate state-of-the-art closed- and open-source multimodal LLMs with various prompting techniques, revealing significant performance gaps. To address these limitations, we propose the Taxonomy-Driven Reflective LLM Chain (TRLC), which boosts detection accuracy by 11.62%.

About the Speaker

Xinyi Zhao is a fourth-year PhD student at the University of Washington, specializing in multimodal large language models and reinforcement learning for smart home applications. This work was conducted during her summer 2024 internship at Wyze Labs, Inc.

Interactive Medical Image Analysis with Concept-based Similarity Reasoning

What if you could tell an AI model exactly “𝘸𝘩𝘦𝘳𝘦 𝘵𝘰 𝘧𝘰𝘤𝘶𝘴” and “𝘸𝘩𝘦𝘳𝘦 𝘵𝘰 𝘪𝘨𝘯𝘰𝘳𝘦” on a medical image? Our work enables radiologists to interactively guide AI models at test time for more transparent and trustworthy decision-making. This paper introduces the novel Concept-based Similarity Reasoning network (CSR), which offers (i) patch-level prototype with intrinsic concept interpretation, and (ii) spatial interactivity. First, the proposed CSR provides localized explanation by grounding prototypes of each concept on image regions. Second, our model introduces novel spatial-level interaction, allowing doctors to engage directly with specific image areas, making it an intuitive and transparent tool for medical imaging.

Paper: Interactive Medical Image Analysis with Concept-based Similarity Reasoning

About the Speaker

Huy Ta is a PhD student at the Australian Institute for Machine Learning, The University of Adelaide, specializing in Explainable and Interactive AI for medical imaging. He brings with him four years of industry experience in medical imaging AI prior to embarking on his doctoral studies.

Multi-view Anomaly Detection: From Static to Probabilistic Modelling

The advent of 3D Gaussian Splatting has revolutionized and re-vitalized the interest in multi-view image data. Applications of these techniques to fields such as anomaly detection have been a logical next step. However, some of the limitations of these models may warrant a return to already applied probabilistic techniques. New approaches, difficulties and possibilities in this field will be explored in this talk.

About the Speaker

Mathis Kruse is a PhD student in the group of Bodo Rosenhahn in Hanover, Germany, where he studies anomaly detection (especially in images). He has a particular interest in multi-view data and its learning-based representations.