Learning when to give up: theory, practice and perspectives

Boston Natural Language Processing Meetup
Boston Natural Language Processing Meetup
Public group

207 Summer St

207 Summer Street · Boston

How to find us

207-215 Summer St, Boston, MA 02210

Location image of event venue


Come join us for our Boston NLP Meet up!

Our special guest is Patrick Haffner (https://www.linkedin.com/in/patrick-haffner-bbb386/) from Interactions

*Pizza and Beer will be provided :)

Learning when to give up: theory, practice and perspectives

The impact of mistakes made by AI systems is becoming increasingly costly and sometimes dangerous. However, even in the case of a simple classifier, standard machine learning techniques only provide us with a probability that this example has been seen in the training data: the system is still unaware of what it does not know. In the context of Conversational AI systems, we will show through examples how critical this ability to say 'I do not know' is, for instance to decide when to transfer to human assist, rely on underspecification, or pick up an example for active labelling. We will then review a wide range of candidate solutions: the theoretically grounded conformal prediction, separate confidence prediction classifiers or more recent Deep Learning architectures such as Adversarial Networks. On spoken language understanding tasks, we will show that high-level domain knowledge and other sources of uncertainty (such as ASR confidence measures) are also essential to reach a rejection decision.

Patrick Haffner has worked on machine learning algorithms since 1988. With Yann LeCun, he was one of the pioneers in applying Neural Networks to speech and image recognition, and led the deployment of the first NN used for an automation task (check reading). With AT&T Labs Research, he was an expert in the learning algorithms that enable data engineers to efficiently train machines using real world data, for tasks ranging from language understanding to network monitoring. He was also an expert advisor to the European Union for their funding programs on machine learning and cognitive sciences. Patrick Haffner is currently a Lead Scientist at Interactions with responsibility for managing the ever increasing variety of machine learning techniques and software that an AI-driven company needs to use.

Directions to Rakuten: