(NY) - Gordon Ritter - The Usefulness of Reinforcement Learning in Finance

The Usefulness of Reinforcement Learning in Finance

Gordon Ritter

Seminar Program

5:45pm Registration
6:00pm Seminar
7:30pm Reception

Abstract

Trading in real markets typically involves planning trades spanning multiple horizons in the future, and properly accounting for trading costs in the trade planning. Under certain assumptions on the statistical process generating returns, one may show that von Neumann-Morgenstern rational decision-makers plan their trades to optimize a utility function of final wealth, where the wealth random variable has all costs included. In a world where markets are not perfectly efficient, could machines learn to optimize utility of wealth? We show that this is indeed possible, by giving an example where trading costs are high, but there nonetheless remains a statistical arbitrage opportunity after costs. A reinforcement learning algorithm learns to trade without knowing a priori that trading costs even exist, and without any a priori model for the stochastic return-generating process. Rather, aspects of the cost function and the return-generating process are encoded in the agent's beliefs concerning the Bellman value function. We discuss the most likely applications of this new technology to markets; in particular, both optimal execution in the style of Almgren and Chriss, and optimal hedging of derivative contracts are special cases of the expected-utility framework, and hence these problems lend themselves to possible handling by agents trained using reinforcement learning. In the option-hedging case, we explain the underlying state-space model and show that the relevant "states" are the same as those considered by Arrow and Debreu.

Biography

Gordon Ritter completed his PhD in mathematical physics at Harvard
University in 2007, where his published work ranged across the fields
of quantum computation, quantum field theory, differential geometry
and abstract algebra. Prior to Harvard he earned his Bachelor's
degree with honours in Mathematics from the University of
Chicago. Prof. Ritter is currently a Professor at NYU, Rutgers, and
the award-winning Baruch MFE program, where his research interests
are focused on portfolio optimization and statistical machine
learning. Prof. Ritter is also a leader in the quantitative trading
industry. He is preparing to launch his own company which will manage
money for institutional clients by means of high-Sharpe pure alpha
systematic trading strategies. He has ten years' experience doing
this; most recently he built a successful trading system from scratch
at GSA Capital, a firm which won the Equity Market Neutral &
Quantitative Strategies category at the Eurohedge awards four
times. Prior to GSA, Gordon was a Vice President of Highbridge Capital
and a core member of the firm's statistical arbitrage group, which
although less than 20 people, was responsible for billions in profit
and trillions of dollars of trades across equities, futures and
options with low correlation to traditional asset classes.

Disclaimer

This a joint IAQF/Thalesians seminar, and not an instructional program of New York University.