How the Humble Regression Became the Next Big Thing

Large Language Models provide incredible text processing and classification abilities. But, as complexity increases, understandability decreases. An LLM could make excellent predictions, but the features which are producing the statistical correlations are unclear. A linear regression, by contrast, may provide a less accurate prediction, but it is trivial to see which features are used to generate the prediction.

In this session, we begin with a corpus of text and a humble regression. From this point, we identify the shortcomings of using this technique and introduce some ways to avoid this. We then iterate through this process, addressing shortcomings while gradually adding sophistication and complexity to the model. Eventually, we arrive with modern LLM architecture with self-attention.

Audience:
This talk is for everyone who wants to understand how we got here, and how LLMs are a natural result of decades of improvements and experiments starting from simple statistical techniques.

Agenda:

1. How the Humble Regression Became the Next Big Thing – Johan van Rooyen

Logistics:

• Format: Hybrid (remotely and in person)
• In Person: Entelect (Boardroom TBC)
• Remote: Now on Teams here (Meeting ID: 363 659 556 021 464, Passcode: ZZ6tu7Ci)
• As usual, pizza and refreshments will be served for those attending in person.