Past Meetup

Counterfactual Learning and Anomaly Detection

This Meetup is past

86 people went

Details

This Friday, we'll have two talks followed by drinks. This week we will have an industry talk by Dr. Stephen Dodson the Tech Lead, Machine Learning at Elastic (https://www.elastic.co) and an academic talk by Artem Grotov from ILPS (http://ilps.science.uva.nl/).

This edition of SEA will be held in SPUI25.

Program:

16:00 - 16:30 Artem Grotov

16:30 - 17:00 Stephen Dodson

17:00 - 18:00 Drinks & Snacks

Details of the talks:

----------------------------------------------------------------------------------------

Artem Grotov -- Deep Counterfactual Learning

Deep learning is increasingly important for training interactive systems such as search engines and recommenders. They are applied to broad a range of tasks, including ranking, text similarity, and classification. Training neural network to perform classification requires a lot of labeled data. While collecting large supervised labeled data sets is expensive and sometimes impossible, for example for personalized tasks, there often is an abundance of logged data collected from user interactions with an existing system. This type of data is called logged bandit feedback and utilizing it is challenging because such data is noisy, biased and incomplete. We propose a learning method, Constrained Conterfactual Risk Minimisation (CCRM), based on counterfactual risk minimization of empirical Bernstein bound to tackle this problem and learn from logged bandit feedback. We evaluate CCRM on an image classification task. We find that CCRM performs well in practice and outperforms existing methods.

Artem Grotov is a PhD candidate supervised by Prof. Maarten de Rijke at the Informatics Institute of the University of Amsterdam. He works on online Learning to Rank as well as other topics that deal with interpreting data obtained from user interactions with interactive systems. His work on click models and evaluating search engines based on logged user interaction have been published at SIGIR 2015 and CLEF 2015.

----------------------------------------------------------------------------------------

Dr. Stephen Dodson -- A Deep-Dive into Anomaly Detection

The talk will outline how machine learning technologies are being integrated into the Elastic Stack, and how these technologies can provide valuable insights into data stored in Elasticsearch now and in future releases.

As well as a high level overview, we will deep-dive into Anomaly Detection. Specifically, we will describe some of the data characteristics which make anomaly detection for real world problems challenging and describe some of the techniques we have developed at Elastic for anomaly detection. As the complexity of IT systems and the quantity of data people gather increases, proactively managing the health and security of these systems requires increasingly sophisticated monitoring tools. Rule based approaches are either becoming unmanageable or in need of augmentation, and the complexity and scale of the data pose significant challenges. Recent techniques from the fields of Data Mining, such as sketch data structures, probabilistic suffix trees, random forests, robust estimation, fitting "fat-tailed" distributions, proper handling of heterogeneous data types, sequential Monte-Carlo and so on, are all useful for improving the quality and/or scalability of anomaly detection. We'll also outline how machine learning technologies are being integrated into the Elastic Stack, and how this technology provides insight into machine data.

Dr Stephen Dodson is Tech Lead, Machine Learning at Elastic. He was previously founder and CTO at Prelert (acquired by Elastic in Sept 2016). Stephen has over 16 years of experience in enterprise systems and software development. He holds a masters in mechanical engineering and a PhD in computational methods from Imperial College, London alongside a CES from École Centrale de Lyon. His academic research focused on computation of large scattering problems using integral equation time domain methods.