Video Transformer Network - A New Approach for Video Action Recognition

We are excited to announce our upcoming meet-up - our first since COVID-19!

The meet-up will feature two sessions run by members of theator’s team. In the first session, Omri Bar will demonstrate how theator utilizes computer vision to build a Surgical Intelligence platform capable of understanding surgical moments based on video footage from previous procedures.

In his talk, Omri will describe how theator’s research team solves problems like surgical workflow step recognition, intraoperative event detection, Out-of-Body detection, and surgical video de-identification. We will explore the different approaches required for each task and how each part of the surgical workflow cycle connects to support offline and online understanding of the complicated surgical setting.

In the second session, Daniel Neimark will present theator’s recent paper on applying Transformers for video action recognition.

We want to thank our sponsors, versatile.ai, for hosting the event and providing the venue and refreshments.

Abstract:
Video action recognition is a foundational task in computer vision. In contrast to recent publications that focus on 3D ConvNets, we present VTN (Video Transformer Network), a transformer-based framework for video action recognition.
The modular structure of VTN first obtains features using a 2D backbone (such as ViT) and then applies a Transformer encoder on top of them. This approach provides two main advantages: firstly, it allows users to process a complete video in a single pass; and secondly, it considerably improves training and inference efficiency by avoiding expensive 3D convolutions.
These advantages make VTN particularly suitable for modeling long videos where interactions between entities are spread throughout the video length.
We will discuss VTN architecture, show the ablation experiments performed on Kinetics-400 and Moment in Time benchmarks, and demonstrate the model’s performance.
Link to the paper: https://arxiv.org/abs/2102.00719

About Omri Bar:
Omri Bar is Head of AI at theator. Omri has over eight years of experience in applying computer vision methods in the medical domain. He holds an M.Sc. in Electrical and Computer Engineering and a B.Sc. in Biomedical Engineering.

About Daniel Neimark:
Daniel is a computer vision engineer at theator. He was previously a team leader in Israel's military intelligence Unit 81. He holds a B.Sc. in Physics and Computer Science from The Hebrew University of Jerusalem.

About theator:
Theator is pioneering the Surgical Intelligence revolution, with an innovative platform built for surgeons by surgeons to address the variability and disparity in surgical care. Combining highly sophisticated artificial intelligence and computer vision technologies, Theator puts defining intraoperative moments in surgeons’ hands so that they can continuously perfect their surgical craft. Theator is partnering with visionary surgeons, hospitals, professional societies, and research institutions to create a smarter, more transparent operating room while empowering surgeons to raise their expertise to the highest possible level. Based in Palo Alto, California, the company is paving the path for real-time surgical decision-support, creating a stronger, healthier world. For more information, visit https://theator.io/.