Tech Lunch Talk with Jannik Kossen

Todays Lunch Tech Talk will be held by Jannik Kossen, a PhD student in machine learning in the OATML Oxford.

He will talk about his recently published paper "Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning"

"We challenge a common assumption underlying most supervised deep learning: that a model makes a prediction depending only on its parameters and the features of a single input. To this end, we introduce a general-purpose deep learning architecture that takes as input the entire dataset instead of processing one datapoint at a time. Our approach uses self-attention to reason about relationships between datapoints explicitly, which can be seen as realizing non-parametric models using parametric attention mechanisms. However, unlike conventional non-parametric models, we let the model learn end-to-end from the data how to make use of other datapoints for prediction."

Self-Attention Between Datapoint is not yet accepted at a venue, available as pre-print at https://arxiv.org/abs/2106.02584