Introduction to Diffusion and Diffusion-LLMs

We will first cover the basics of diffusion, the process by which we gradually add noise to structured data until pure noise is achieved, and how to reverse it back to a clean image with the help of deep learning models. Diffusion is most prominent with image generation, where we start from a noisy image and leverage our model to generate a clean image of our choosing. But can we take it further, by applying this same diffusion philosophy to generate a full, coherent LLM response simultaneously? We dive into how this can be achieved, covering both theoretical and implementation details, the advantages diffusion-LLMs have over traditional LLMs, and their limitations.