addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupshelp-with-circleimageimagesinstagramlinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1ShapeoutlookpersonJoin Group on CardStartprice-ribbonShapeShapeShapeShapeImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruserwarningyahoo

Thalesians/QFGG (Frankfurt) - Thomas Wiecki - Predict out-of-sample performance

  • May 11, 2016 · 6:30 PM
  • This location is shown only to members

NOTE: the event is at 6.30pm Frankfurt time (please ignore the Outlook invite time)

Full title: Thomas Wiecki - Predicting out-of-sample performance and building multi-strategy portfolios using Random Forests

Thanks for Jochen Papenbrock and Adrian Zymolka for organising and for PPI AG for hosting. Tickets will be FREE for this event!

You can access the Thalesians/Quant Finance Germany (Frankfurt) LinkedIn Group page here.


The question of how predictive a backtest is of out-of-sample performance is at the heart of algorithmic trading. Using a unique dataset of 888 algorithmic trading strategies developed and backtested on the Quantopian platform with at least 6 months of out-of-sample performance, we study the prevalence and impact of backtest overfitting. Specifically, we find that commonly reported backtest evaluation metrics like the Sharpe ratio offer little value in predicting out of sample performance (R² < 0.025). However, we show that by training a Random Forest regressor on a variety of features that describe backtest behavior, out-of-sample performance can be predicted at a much higher accuracy (R² = 0.17) on hold-out data compared to using linear, univariate features. We then show that we can construct a multi-strategy portfolio based on predictions by the Random Forest which performed significantly better out-of-sample than other alternatives.

Speaker: Thomas Wiecki is the Data Science Lead at Quantopian focusing Bayesian models to evaluate trading algorithms. Previously, he was a Quantitative Researcher at Quantopian developing an open-source trading simulator as well as optimization methods for trading algorithms.

Thomas holds a PhD from Brown University.

Join or login to comment.

  • Saeed A.

    A message from Thales, Jochen and Adrian!

    Thanks for joining the Thalesians Frankfurt/Quant Finance Group Germany seminar on May 11
    with Thomas Wiecki presenting on “Predicting out-of-sample performance and building multi-strategy portfolios using Random Forests” We had an exciting session and a lively discussion yesterday at the Thalesians seminar – thanks everyone for participating and contributing!

    Thanks also to PPI for providing the superior view seminar room and to Axioma for sponsoring the drinks. And of course - special thanks to Thomas for his great presentation!

    Thomas has also provided some materials for those interested (scroll down page for these!) And please feel free to twitter your personal thank you to Thomas: We are looking forward to our next seminar in summer/fall which will be announced here – stay tuned!

    May 13

  • Peter S.

    I also would like to thank Adrian and Jochen for the organization! Thomas, good talk! I already told a lot of people about Quantopia and the idea behind it.

    Cheers, see you next time,

    2 · May 13

  • Matthias R.

    Thanks, Thomas, for the great talk. Would definitely love to hear how the results develop with growing OOS data.

    2 · May 12

  • Levent

    Hi Jochen, hi Adrian - thank you very much for organizing the meeting. Great presentation Thomas!

    2 · May 12

  • Thomas

    Many thanks for coming out last night and the many interesting questions and discussions. I enjoyed meeting every single one.

    Here are the slides of my talk:

    Link to the paper (submitted):

    And last but not least, check out and let me know what you think!

    3 · May 12

    • Jochen P.

      Thanks a lot Thomas. It was an exciting session. Everybody enjoyed. Please come by in one of our next sessions and report updates.

      1 · May 12

  • Arvind

    Great talk:) very interesting results.

    4 · May 12

  • Dr. Pawel L.

    Thomas, will your presentation be available online somehow sometime somewhere? Thanks! -Pawel

    April 20

Our Sponsors

  • MathFinance

    The bridge between investment banking and academic research

  • MoneyScience

    Financial Intelligence for the Business World

  • Supermicro UK

    High-Performance Server Solutions

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy