Meetup #13: Practical Data Science | The Transformer

Dies ist ein vergangenes Event

71 Personen haben teilgenommen

Bild des Veranstaltungsortes

Details

The next edition of the meetup brings you yet another evening packed with people working on cool stuff all around data. Mark and Tim share an approach to cover everything your data science project needs. And Andreas tells us how and why the Transformer architecture beats RNN.

We are hosted by the Venture Club Münster and TechLabs in their hive in the middle of the town. Be there or be square!

----------------------
"Practical data science – How to track your development process with DVC"
Mark Keinhörster & Tim Sabsch

Mark is Data Plumber at codecentric in Münster. He enjoys putting data-science code to production.
Tim started his journey as a Data Scientist at codecentric Münster. He currently reads into DataOps technologies and Generative Models.

Abstract: Datacentric applications utilising machine learning models have evolved into common solutions. Many projects however still suffer from a lack of good patterns and practices, when developing such powerful technologies. Digging down into the nitty-gritty details, we explain how you can use DVC to version all parts of your projects: From the dataset, over gluecode up to the model itself. But wait, there's more! We show you code that covers the full development cycle, including experiments and reproducability, as well as release and deployment of your model to machines in the wild.

----------------------
"The Transformer - Sequence Transduction with Self-Attention"
Andreas Köpf

Andreas is a managing partner and software architect at PROVISIO GmbH. During the last three years he led the internal adaptive robotics group “Xamla” that developed the robot programming system Rosvita for lab-automation tasks. He is an early PyTorch contributor who helped modernizing the Torch7 C-backend for use from Python.

Abstract: We take a closer look at one of the most popular Deep Learning models for sequence modeling: The Transformer.

Sequence modeling is used for a variety of tasks like translation, summarization and text generation. A major milestone in this field was Google's Transformer architecture ("Attention Is All You Need", Vaswani et al., 2017). Nowadays Transformer blocks are the foundation of powerful language models like BERT, GPT2 and Transformer-XL. They can also be used in generative models with applications beyond Natural Language Processing, e.g. for audio synthesis.

The talk will give you an overview of the Transformer architecture and explain why it beats RNN based models with shorter training time and better model quality. A focus will be on the magical "Multi-Head Scaled Dot-Product Self-Attention" mechanism and its history. But we will also drill down into some details of input embeddings, positional encodings, normalization, masking and output sampling. The talk does not require any previous NLP knowledge.

See you soon,
Tobias²