Your Backtest Is Lying to You: Look-Ahead Bias and Data Leakage in Financial ML

# Your Backtest Is Lying to You: Look-Ahead Bias and Data Leakage in Financial ML

### Out of Sample 1 | A Live Python Workshop

📍 Location: Online—event link provided to registered attendees
📅 Date: Thursday, November 12, 2026
🕡 Time: 6:30–8:00 PM ET
⏱️ Duration: 90 minutes
🎟️ Admission: $35 USD Rsvp and pay here
👥 Hosted by: NYC Quants and Data Scientists

## About This Event

Your model performs well on historical data—but does your evaluation reflect how it would make predictions in the real world?
Join the first session of Out of Sample, a practical workshop series exploring quantitative finance, data science, and machine learning.

In this 90-minute session, I’ll examine how data leakage and look-ahead bias can undermine financial model evaluation. Through practical examples and a live Python demonstration, we’ll compare the same features and machine learning model using two approaches: shuffled K-fold cross-validation and walk-forward validation.
I’ll interpret the results together, discuss what each approach actually measures, and identify checks you can apply to your own projects.

This is a focused, demonstration-led workshop.

## Your Takeaways

You’ll leave with practical tools for reviewing financial machine learning experiments:

• A reusable Python notebook comparing shuffled K-fold and walk-forward validation, available through a public GitHub repository.
• A backtest review checklist covering feature timing, training/test separation, preprocessing, and overlapping prediction horizons.
• A clearer understanding of look-ahead bias and data leakage, supported by examples discussed during the session.
• A framework for interpreting validation results: what the scores tell you, what they do not tell you, and what to investigate next.

## The 90-Minute Agenda

### 6:30–6:40 PM | The Backtesting Problem

We’ll introduce the prediction task and establish the central question: what information would actually have been available when each prediction was made?
You’ll learn to distinguish a strong historical score from evidence that a model has been evaluated appropriately.

### 6:40–7:00 PM | How Leakage Enters an Experiment

We’ll examine examples of future information entering features, preprocessing, or model evaluation, and discuss why prediction horizons matter when separating training and test data.
You’ll practice asking the key question: “Could I have known this at prediction time?”

### 7:00–7:30 PM | Live Python: Compare Two Validation Approaches

We’ll evaluate the same features and model using shuffled K-fold cross-validation and walk-forward validation, then compare and interpret the results.
You’ll see how the evaluation setup changes the question being answered—and why changing the split alone does not resolve every source of leakage.

### 7:30–7:45 PM | Build Your Backtest Review Checklist

We’ll turn the demonstration into a short, repeatable review process for your own work. We’ll also briefly introduce when overlapping labels may require additional safeguards, including purging and embargoing.
You’ll leave with specific checks to run before trusting a performance metric.

### 7:45–8:00 PM | Live Q&A

Ask questions about the demonstration, validation choices, and how to apply the session’s principles to your own projects.

## Who Should Attend?

• Data scientists working with time-dependent financial data.
• Quantitative researchers and analysts reviewing predictive models.
• Machine learning engineers interested in financial applications.
• Students and aspiring quants with basic Python and machine learning knowledge.

## Prerequisites and Format

Basic familiarity with Python, model training, and train/test splits is recommended.
The session is demonstration-led, so you can attend without running code live. The demonstration notebook will be available through the public GitHub repository for further exploration.

## About Out of Sample

Out of Sample is a workshop series from NYC Quants and Data Scientists, focused on practical demonstrations and thoughtful discussions at the intersection of quantitative finance, data science, and machine learning.
Join us for our inaugural session to learn how to question your backtest—not just celebrate its score.