[online] Abu Dhabi Machine Learning Meetup Season 2 Episode 4

Talk 1: Ensuring Safety for Reinforcement Learning

Constrained reinforcement learning (CRL) agents can trade off performance and safety autonomously based on the constrained Markov decision process (CMDP) framework since it decouples safety from reward. Unfortunately, most CRL agents only guarantee safety after the learning phase. In this talk, we present two approaches that can enhance the safety of an RL agent.

The first setting provides a concise abstract model of the safety aspects, a reasonable assumption since a thorough understanding of safety-related matters is a prerequisite for deploying CRL in typical applications. We propose a CRL algorithm that uses this abstract model to learn policies for CMDPs safely, that is, without violating the constraints during the learning phase.
In the second setting, the agent learns to behave safely without a goal in a simulated/controlled environment, which allows unsafe interactions and provides the safety signal.
Eventually, this agent is deployed in a target task, where it has a goal and safety violations are not allowed anymore.
We draw from the transfer learning framework to train a new policy for the target environment without violating the safety constraints using the policy from the initial environment.

Speaker: Thiago is a PostDoc researcher at Radboud University Nijmegen advised by Dr. Nils Jansen. Previously, he was is a Ph.D. candidate within the Algorithmics Group at Delft University of Technology, advised by Dr. Matthijs Spaan. His research interests lie primarily in the automation of sequential decision-making, focusing on reinforcement learning and its safety aspects. He obtained his M.Sc. degree in artificial intelligence from the Instituto de Matemática e Estatística at Universidade de São Paulo under the supervision of Dr. Leliane N. de Barros and a bachelor degree in computer science at the Departamento de Ciência da Computação at Universidade Federal de Lavras.

More information at https://tdsimao.github.io

Talk 2: Diversity in Generative models

GANs are notoriously difficult to train.
In this talk we will present score-based (denoising diffusion) generative models that have the same generative abilities but with a much lower computational cost. We will also discuss the implications of mode collapse in such diffusive based models, the pitfalls and some ideas on how to tackle them.

Talk reported to next meetup: Machine Learning in Pairs Trading

Speaker: Illya Barziy, Quant Research Team Lead at Hudson & Thames https://hudsonthames.org

Abstract: In this talk, we'll be covering a novel application of Machine Learning in the Pairs Trading domain. The new framework is based on PCA and DBSCAN/OPTICS clustering. It allows discovering well-performing pairs of assets from a big pool of securities, easily overseen by classic pairs selection methods. Pairs Trading or Statistical Arbitrage is a broad group of trading strategies that evolved over the last 20 years and can be split into more than five big classes, depending on the nature of the strategy. Usually, Machine Learning is used to predict the performance of the spread (difference in prices of a pair of assets) and convert the information to a trading signal.