[online] Hong Kong Machine Learning Meetup Season 3 Episode 12

Last meetup of the season 3.
Hopefully, for season 4 we will be able to organize events that are both IRL and online. Stay tuned!

Talk 1: Bayesian Modeling without the Math

Alexandre Andorra, Principal Data Scientist & Co-founder, PyMC Labs
https://www.pymc-labs.io/
https://github.com/AlexAndorra

Talk 2: Julien Launay & Igor Carron, LightOn https://lighton.ai/

PAGnol

Extreme-scale models with billions of parameters (GPT-3, T5, etc.) have garnered increased interest over the past year, demonstrating unique few-shot learning abilities.
We introduce PAGnol, a collection of French GPT models. Using scaling laws, we efficiently train PAGnol-XL (1.5B parameters) with the same computational budget as CamemBERT, a model 13 times smaller. PAGnol-XL is the largest model trained to date for the French language.
We also explore some of the motivations behind extreme-scale models: from the surprising finding of scaling laws that larger models are more efficient, to recent research on prompt-tuning questioning the established fine-tuning paradigm.