Skip to content

Details

Join us for our 26th Meetup! This time on the topic “Solving Problems Without Training (Full) Models” featuring speakers Doron Haritan, Data Scientist at NVIDIA and Yoel Zeldes, Algorithm Engineer at AI21 Labs.

*For full zoom details & password, please register here: https://airtable.com/shrBx1XHQvqpCLRwq

This event will also be live streamed on the Made in Jerusalem Facebook page - facebook.com/madeinjlm.

The talks we will hear are “A Review on Black-Box Optimization Methods” and “Auxiliary Tuning and its Application to Conditional Text Generation”.
Talk 1: “A Review on Black-Box Optimization Methods”
Black-box optimization deals with cases where the transformation from input to output is unknown while the sampling of the input-output pair is possible. In network devices, for example, the device complexity is beyond analytical formulation, but the input-output pairs are available. By using black-box optimization methods, we can improve the performance of the device. How is it possible without knowing the transformation function? Which optimization methods can we use? In this talk, Doron will review a selection of derivative-based optimization methods and compare them to the deviate free optimization methods, given the production constraints.

Talk 2: “Auxiliary Tuning and its Application to Conditional Text Generation”
AI21 Labs has developed a new method called Auxiliary Tuning that adapts a pre-trained language model (LM) to a novel language task. Let's say that you want to train a model to generate text conditioned on a control input (e.g. sentiment). Training it from scratch can be costly, since SOTA performance and fluency tend to correlate with model size. We'd like to leverage what the LM (e.g. GPT-2/GPT-3) has learned, and somehow make it consider the control input. Enters Auxiliary Tuning: while the LM wasn't trained to handle the control input, you can train an auxiliary model that takes the control input as an additional input and outputs logits. All you have to do is to add the logits to the LM's logits. Surprisingly, this yields competitive results to training from scratch but using significantly less compute.

Schedule:
17:50-17:55: Signing in

17:55-18:00: Opening words & announcements

18:00-18:30: Talk 1: “A Review on Black-Box Optimization Methods”

18:30-19:00: Talk 2: “Auxiliary Tuning and its Application to Conditional Text Generation”

19:00-19:15: Q&A from viewers on both talks

Facebook event link: https://www.facebook.com/events/3453390618092130

The JerusML Webinars are sponsored by:
Start-Up Nation Central
Made in JLM
JLM.3K
The Jerusalem Foundation

You may also like