Ofir Press | Complementing Scale: Novel Guidance Methods for Improving LMs

Virtual London Machine Learning Meetup.

Title: Complementing Scale: Novel Guidance Methods for Improving Language Models

Speaker: Ofir Press (deep learning researcher and recent doctoral graduate from the University of Washington)
Abstract: This talk will cover a few of my recent papers, and will discuss my current views on the field and what future directions excite me.
First I'll provide a quick overview of ALiBi, and talk about how to build LMs that can process longer sequences than those they were trained on.
I'll talk about how to evaluate such models and my thoughts about all the recent LLaMA extrapolation research.
If you are interested in an in-depth lecture on ALiBi you can watch a previous talk just on that here ( https://www.youtube.com/watch?v=Pp61ShI9VGc ), but no worries if not - I will provide a summary.
Then I will focus on our self-ask method, which is a prompting method that improves over chain-of-thought and allows easy integration between LM and search engines.
If time permits I will discuss two future directions that I believe the community should focus on.
I'm very open to questions and will allocate my time based on audience interest.

​Speaker bio: I recently received my PhD from the Paul G. Allen School for Computer Science & Engineering at the University of Washington, where I was very fortunate to be advised by Noah Smith.
During my PhD I spent two years as a visiting researcher at Facebook AI Research Labs on Luke Zettlemoyer’s team where I mainly worked with Mike Lewis. Prior to that, in the summer of 2019 I interned at Facebook AI Research with Omer Levy.
Towards the end of my PhD I spent half a year as a visiting researcher at MosaicML on Jonathan Frankle’s team.

Agenda:

• 18:25: Virtual doors open
• 18:30: Talk
• 19:10: Q&A session
• 19:30: Close

Sponsor: Evolution AI - Intelligent data extraction from corporate and financial documents.