Designing Your Neural Networks with Weights and Biases: A Walkthrough

Neural networks are powerful beasts that give you a lot of levers to tweak! The sheer size of customizations that they offer can be overwhelming to even seasoned practitioners. In this talk I’ll give you a framework for making smart decisions about your neural network architecture!

We’ll explore lots of different facets of neural networks in this talk, including how to setup a basic neural network (including choosing the number of hidden layers, hidden neurons, batch sizes etc.) We’ll learn about the role momentum and learning rates play in influencing model performance. And finally we’ll explore the problem of vanishing gradients and how to tackle it using non-saturating activation functions, BatchNorm, better weight initialization techniques and early stopping.

I hope this guide will serve as a good starting point in your adventures.

Bio:

Lavanya is a machine learning engineer @ Weights and Biases. She began working on AI 10 years ago when she founded ACM SIGAI at Purdue University as a sophomore. In a past life, she taught herself to code at age 10, and built her first startup at 14. She's driven by a deep desire to understand the universe around us better by using machine learning. You can find her on Twitter: @lavanyaai

Talk: https://lavanya.ai/2019/08/10/training-a-neural-network-start-here/
Code: https://github.com/lavanyashukla/neural_networks