How AI Learns Like an Octopus: Conquering High-Dimensional Action Spaces

An octopus has eight arms that can move in countless ways. Coordinating all these arms presents a significant challenge due to the vast number of possible movements, too complex for a single control system. Turns out nature has a clever solution, by granting each arm a degree of autonomy, allowing it to make its own decisions.

A similar problem arises in reinforcement learning, where too many action possibilities can hinder performance or even be unsolvable with a single control. The paper titled "Action Branching Architectures for Deep Reinforcement Learning" proposes the Branching Duelling Q-Network (BDQ), a method to address the overwhelming action possibilities by branching them into multiple dimensions.

Join us for a thorough discussion to explore how effective this natural strategy translates to reinforcement learning problems. We will touch on concepts of reinforcement learning, how the BDQ works, and what practical applications this method offers.

Paper link: https://arxiv.org/abs/1711.08946