Action Robust Reinforcement Learning and CNN on Manifolds

Agenda:
18:00-18:20: Gathering and networking
18:20-18:30: Opening words and Augury introduction
18:30-19:15: Action Robust Reinforcement Learning and Applications in Continuous Control, by Chen Tessler.
19:15-20:00: CNN on Manifolds, by Dr. Uri Itai

Action Robust Reinforcement Learning and Applications in Continuous Control

Abstract:
In this work we formalize two new criteria of robustness to action uncertainty. Specifically, we consider two scenarios in which the agent attempts to perform an action a, and (i) with probability α, an alternative adversarial action a¯ is taken, or (ii) an adversary adds a perturbation to the selected action in the case of continuous action space. We show that our criteria are related to common forms of uncertainty in robotics domains, such as the occurrence of abrupt forces, and suggest algorithms in the tabular case. Building on the suggested algorithms, we generalize our approach to deep reinforcement learning (DRL) and provide extensive experiments in the various MuJoCo domains. Our experiments show that not only does our approach produce robust policies, but it also improves the performance in the absence of perturbations. This generalization indicates that action-robustness can be thought of as implicit regularization in RL problems.
Chen will also give a brief overview of a work he recently submitted to NeurIPS.

Chen Tessler is a Phd student in the reinforcement learning lab at the Technion supervised by prof. Shie Mannor.

CNN on Manifolds

Abstract:
Convolution on the data is the fundamental operation in deep-cnn. Most of the data we get, can be consider as Euclidian space. However, what if it does not make sense? Or a different geometry structure exists?
For example, when the data is in a graph structure (vertices and nodes), like in e-commerce problems or social networks data. Moreover, in such non-Euclidian geometry what is the meaning of the convolution operation?
Apparently, it is possible to define a generalization of the convolution operation on geometric manifolds. In this talk I would present method to generalize the convolution operator to non-Euclidean manifolds.

Uri Itai has a PhD in applied mathematics from the Technion (IIT) under the supervision of Prof. Nira Dyn. His dissertation was the study of constructing surfaces for non-Euclidean manifolds.
After graduating he was working in various fields of data science, algorithmic trading, cancer research and more. Today he is a senior researcher in SafeRide. A start up in the field of cyber for autonomous cars.

******
Augury is building a world where people can always rely on the machines that matter. Augury supports its partners by enabling Digital Transformation through superior insights into the health and performance of the machines they use to make products, deliver services and improve lives.

AIFortify is an early stage startup that develops AI placement technologies.