Structured Dynamics Inference through Kinematic In-Hand Manipulation

In-hand manipulation—the task of changing an object’s pose with respect to the robot’s palm, remains challenging to perform in the unstructured real world. Current methods for in-hand manipulation either require extensive knowledge of the object or experience with the object to perform the task. Approaches that do not require object knowledge have only shown to generalize to a single object on different targets (goal states) or single target on different objects. The first part of my talk will focus on in-hand manipulation of objects with unknown dynamics. Specifically, I will discuss our novel analytic formulation of the in-hand manipulation task, enabling our approach to perform on different objects on different targets without requiring knowledge of object dynamics.

The second part of my talk will explore inferring force during manipulation given knowledge of object dynamics. We leverage this force inference to learn a model to estimate force from the BioTac sensor, a commonly used tactile sensor for manipulation. Our novel neural network architecture encodes the geometry of the sensor in 3D convolutional neural networks. A key aspect of our approach is leveraging robots to collect a large scale dataset. This research was a Finalist for the Best Manipulation Paper award at the IEEE International Conference on Robotics and Automation (ICRA) 2019. I will conclude the talk with some results from our ongoing research on combining the above two directions for gaining knowledge about unknown objects in the physical world.

Balakumar Sundaralingam is a Ph.D. candidate at the University of Utah, affiliated to the School of Computing and the Utah Robotics Center. His research on robot manipulation has led to multiple publications at top tier robotics conferences. He is a full stack roboticist, with over 7 years of experience in building autonomous robotic systems. In Summer of 2018, he interned at NVIDIA Seattle robotics lab. During his internship, he worked on research projects related to perception for robot manipulation and also developed software tools for robotic systems.