We are very happy to announce the 10th PyData Cambridge meetup!

Many thanks to Raspberry Pi, who host the group.

Agenda

18:45 - Doors open
19:00 Introduction
19:05 "Lightning talks" by PyData Cambridge members (you!)
19:45 - Interval / pizza and drinks provided
20:15 - “Modelling human uncertainty: how to teach machines when experts disagree with each other” by Ryutaro Tanno (abstract below)
21:00 - End (Pub TBA)

Code of Conduct

PyData is dedicated to providing a harassment-free event experience for everyone, regardless of gender, sexual orientation, gender identity, and expression, disability, physical appearance, body size, race, or religion. We do not tolerate harassment of participants in any form.

The PyData Code of Conduct governs this meetup. ( http://pydata.org/code-of-conduct.html ) To discuss any issues or concerns relating to the code of conduct or the behavior of anyone at a PyData meetup, please contact NumFOCUS Executive Director Leah Silen (leah@numfocus.org) or organizers.

Talks

"Lightning talks" by PyData Cambridge members

Do you want to promote a project you are developing? a cool library or dataset you recently discovered? an experiment you did, that went terribly wrong? maybe a talk that you recently watched and think it is a 'must see'? This is your opportunity! We are hosting a lightning talks session (up to 5 minutes/talk).

This is a great chance for first time speakers, to be in a welcoming environment, get feedback and foster discussion. Everybody is welcome!

Note: if you want to guarantee a slot, please send us an e-mail at pydatacambridge@gmail.com so that we can reserve one for you.

"Modelling human uncertainty: how to teach machines when experts disagree with each other" by Ryutaro Tanno

The predictive performance of supervised learning algorithms depends on the quality of labels. In a typical label collection process, multiple annotators provide subjective noisy estimates of the “truth” under the influence of their varying skill-levels and biases. Blindly treating these noisy labels as the ground truth limits the accuracy of learning algorithms in the presence of strong disagreement.

This problem is critical for applications in domains such as medical imaging where both the annotation cost and inter-observer variability are high. In this work, we present a method for simultaneously learning the individual annotator model and the underlying true label distribution, using only noisy observations. Each annotator is modeled by a confusion matrix that is jointly estimated along with the classifier predictions. We propose to add a regularisation term to the loss function that encourages convergence to the true annotator confusion matrix.

We provide a theoretical argument as to how the regularisation is essential to our approach both for the case of single annotator and multiple annotators. Despite the simplicity of the idea, experiments on image classification tasks with both simulated and real labels show that our method either outperforms or performs on par with the state-of-the-art methods and is capable of estimating the skills of annotators even with a single label available per image.

Short bio: Ryutaro Tanno is a 3rd year PhD student at UCL on a Microsoft Research scholarship. After completing MASt in Mathematics, and MPhil from Computational and Biological Learning group in University of Cambridge, he started his Phd in 2015 under the supervision of Daniel C. Alexander at University College London and Antonio Criminisi of Microsoft Research Cambridge. His main interest lies in developing high-performance machine learning algorithms which are more interpretable and safer to use in healthcare applications. He received a best paper award in MICCAI 2017, the largest international conference on machine learning for medical imaging applications.

---
Many thanks to our sponsors: ARM, Enthought, fetch.ai, Illumina, NumFOCUS and Raspberry Pi.