Skip to content

Details

Talk-Titel: Open Foundation Models: Scaling Laws and Generalisation

Speaker: Dr. Jenia Jitsev

Abstract: To study transferable learning and generalisation, the derivation of reproducible scaling laws is crucial, as they not only predict model
function and properties on unseen scales, but also have potential to
enable systematic comparison of various learning procedures. The talk
will discuss why open foundation models and datasets are essential for
this research and highlight challenges in properly measuring generalisation.

Bio: Jenia Jitsev is co-founder and scientific lead of LAION e.V, the German non-profit research organization committed to research on open
large-scale foundation models and datasets. He also leads Scalable
Learning & Multi-Purpose AI (SLAMPAI) lab at Juelich Supercomputer
Center, Research Center Juelich, Helmholtz Association, Germany and is a
member of ELLIS. His background is in machine learning and neuroscience,
aiming to understand learning as a generic process of incrementally
building up a useful model of the surrounding world from available
sensory observations and executed actions. His current research focus is
on using scaling laws for measuring and understanding generalization and
strong transfer in open foundation models. Jenia is most known for his
work on open language-vision foundation models like openCLIP and open
datasets like LAION-400M/5B, Re-LAION, DataComp. Recently, he also has been studying reasoning and measuring generalization with works on open reasoning datasets/models OpenThouhts/OpenThinker and on discovering generalization weaknesses using AIW problems. Jenia coordinates acquisition of large-scale compute grants for conducting collaborative research on open foundation models across various supercomputing facilities, including EuroHPC. Using these resources, together with the community he is driving and democratizing research on scalable systems for generalist, transferable multi-modal learning, leading to foundation AI models capable of strong transfer and therefore easily adaptable to a broad range of desired tasks and hardware resource settings. He is currently involved in major efforts to establish strong open foundation model research and development track on EU grounds within EU projects openEuroLLM, ELLIOT and MINERVA. For his work, Dr. Jitsev received Best Paper Award at IJCNN 2012, Outstanding Paper Award at NeurIPS 2022 and Falling Walls Scientific Breakthrough of the Year 2023 Award.

We are with IMBIT this time and meet in the Nexus Lab. Averbis provides beers and bezels after the talk, and we invite you to stay for a chat.
Thanks for the support from Averbis and IMBIT!

Events in Freiburg im Breisgau
Artificial Intelligence
Deep Learning
Machine Learning
Robotics
Computer Science

Members are also interested in