#26 - 2019.09 - Google and NLP: From transformers to BERT in Colab

🆂🅻🅾🆃 #1: A deep dive into some modern NLP architectures and techniques: Attention, Transformers, BERT, with hands-on Colabs for them to run on Cloud GPUs / TPUs - Cesar Ilharco, Google, Google Research & Machine Intelligence

TensorFlow will be used for a hands-on experience with modern Natural Language Processing, starting with a practical introduction to Attention and Neural Machine Translation and followed by Transformers and BERT. Participants will get exposed to state-of-the-art NLP architectures and techniques, understand them conceptually and apply them to practical problems, for instance by training Transformers models and fine-tuning pertained BERT with Colab running on Cloud GPUs/TPUs.

Cesar has been working as a software engineer at Google Research & Machine Intelligence / Google Assistant / News Intelligence and Realtime Event Understanding in Zurich, where NLP plays important roles. Jointly, he co-organizes the Zurich NLP Meetup and he has been teaching Deep Learning for Natural Language Processing to Data Science students, covering among other topics: TensorFlow; Optimisation; Neural Networks; Language Models; Embeddings; Machine Translation; LSTMs; Transformers and BERT. Prior to that, Cesar studied Applied Mathematics and Computer Science, and got exposed to Software Engineering / Machine Learning through internships at Google, Amazon Web Services, Facebook and Quora.

🆂🅻🅾🆃 #2: Practical NLP in Google Colab

We continue with theory from above with applied examples how to get your hands dirty

🆂🅻🅾🆃 #3: Practical Advice for real world NLP projects with BERT - Peter Albert

The talk goes through learnings on a recent NLP project and a gives advice on each step of BERT training and deployment. It will go through examples on how to decide on a label schema, how to get started with only a few hundred labeled examples and how to incorporate additional context into BERT. It will demonstrate a method to quickly determine how much data is needed for a certain model accuracy and review options to deploy BERT with low latency.

After graduating from the university, Peter has founded an e-commerce company. In free time, he developed several deep learning side projects from future video frame prediction to building a question-answering system. Currently, he is working on a new startup in a domain of NLP.

This event is jointly organized with GDG Cloud Munich - https://www.meetup.com/GDG-cloud-munich/

We are looking for speakers and sponsors!

As always, we are super happy to have you with a lightning talk on your small or big successes, open source projects or just something amazing you would like to share.

Do not miss out updates on twitter - https://twitter.com/hack_ai and join Munich ML community on Slack channel https://ai-hack-inviter.herokuapp.com/