13. LightOn AI Meetup: SLIDE & MONGOOSE: Efficient Neural Network Training

LightOn is a DeepTech company creating hardware for new AI applications requiring a massive amount of data (molecular dynamics, fraud detection, conversational AI ...). We gather experts, engineers, researchers, who are shaping the world of tomorrow. We are back for our 13th edition!

Our meetups will happen online in order to keep our attendees safe until further notice.

This event takes place at 16:00, Paris time (GMT +02:00)

A Zoom link will be provided prior to the meetup.

---
Agenda

16:00 – Introduction by LightOn

---

16:10 – "SLIDE & MONGOOSE: LSH Frameworks for Efficient Neural Network Training"
by Beidi Chen, Postdoc Researcher at Stanford.

Abstract: With the exponential growth in data and model size, it is necessary to reduce the computational and memory bottlenecks in neural network (NN) training. In this talk, we introduce our first framework SLIDE (Sub-LInear Deep learning Engine), which uniquely blends smart randomized algorithms, Locality Sensitive Hashing (LSH), with multi-core parallelism and workload optimization for efficient NN training. Our evaluations on industry-scale recommendation datasets, with large fully connected architectures, show that training with SLIDE on a 44 core CPU is more than 3.5 times (1-hour vs. 3.5 hours) faster than the same network trained using TF on Tesla V100. Then we present MONGOOSE, a learnable LSH framework equipped with a scheduler and low-cost learnable LSH hash functions. Finally, we demonstrate its further improvements -- up to 8% better accuracy with 6.5x speed-up and 6x reduction in memory usage, in a wide range of tasks.

16:30 - Q&A

---
During the event, you can share content using #LightOn & @LightOnIO

After the meet-up:

• the slides will be available on our Medium page: https://medium.com/@LightOnIO
• the video of the presentation will be available on our Youtube Channel: https://www.youtube.com/channel/UCE6xfSWQaCk1hqu7pCISK1w