Neural Networks: Zero to Hero - Building a WaveNet.

Building makemore Part 4: Becoming a Backprop Ninja

We following a course created by Andrej Karpathy on building neural networks, from scratch, in code.

We start with the basics of backpropagation and build up to modern deep neural networks, like GPT. In my opinion language models are an excellent place to learn deep learning, even if your intention is to eventually go to other areas like computer vision because most of what you learn will be immediately transferable. This is why we dive into and focus on languade models.

Prerequisites: solid programming (Python), intro-level math (e.g. derivative, gaussian).

---

This is sixth event from this series.

We take the 2-layer MLP from previous video and make it deeper with a tree-like structure, arriving at a convolutional neural network architecture similar to the WaveNet (2016) from DeepMind. In the WaveNet paper, the same hierarchical architecture is implemented more efficiently using causal dilated convolutions (not yet covered). Along the way we get a better sense of torch.nn and what it is and how it works under the hood, and what a typical deep learning development process looks like (a lot of reading of documentation, keeping track of multidimensional tensor shapes, moving between jupyter notebooks and repository code, ...).

If you was absent on previous you can watch all previous lessons.

Full curse is presented on page:
https://karpathy.ai/zero-to-hero.html

---

✅ We will follow course program, discuss code, explain fragments that are unclear and learn together.
✅ After course there will be time to eat dinner making new connections in AI worlds and sharing what you think.

Basically you can just view video at your home, but learning in group you can grasp some ideas faster asking questions and learn deeper explaining something that you understand to others.