Reproducibility in AI and How to Improve It With Code Ocean

Hey Data Explorers!

This time we take a look at a serious issue in data science research: reproducibility.

A 2016 survey [1] in the journal Nature showed that more than 70% of researchers have tried and failed to reproduce another scientist’s experiments, and more than half have failed to reproduce their own experiments. Unfortunately data science is no exception to this "reproducibility crisis". In a recent 2019 AAAI Workshop on Reproducible AI [2], a study showed that researchers spent nearly 25 hrs on average to reproduce the results when implementing the algorithm using the same code and same data from original authors, and much more time when trying to implement the method using an alternative approach on the same data.

In this talk, we will discuss the challenges of reproducibility in data science research, and we will learn how to organize and publish reproducible projects using Code Ocean, an in-browser tool designed for computational reproducibility.

Schedule:

6:00-6:30pm Sign in, Network and Food :)
6:30-7:30pm Talk
7:30-8pm Q&A

Speaker:

Xu Fei

Xu Fei is an outreach scientist at Code Ocean (https://codeocean.com), where he spends time to help researchers do reproducible work.

Requirement:

Please bring a laptop if you want to follow along the demo :)

1. https://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970

2. https://www.idi.ntnu.no/~odderik/RAI-2019/presentations/how_can_we_know.pdf