Hugo speaks with Sebastian Raschka, a machine learning & AI researcher, programmer, and author. As Staff Research Engineer at Lightning AI, he focuses on the intersection of AI research, software development, and large language models (LLMs).

How do you build LLMs? How can you use them, both in prototype and production settings? What are the building blocks you need to know about?

In this episode, we’ll tell you everything you need to know about LLMs, but were too afraid to ask: from covering the entire LLM lifecycle, what type of skills you need to work with them, what type of resources and hardware, prompt engineering vs fine-tuning vs RAG, how to build an LLM from scratch, and much more.

The idea here is not that you’ll need to use an LLM you’ve built from scratch, but that we’ll learn a lot about LLMs and how to use them in the process.

Near the end we also did some live coding to fine-tune GPT-2 in order to create a spam classifier!

LINKS

Episode 26: Developing and Training LLMs From Scratch

Vanishing Gradients