Stanford CS229 I Machine Learning I Building Large Language Models (LLMs)

This podcast focuses on the practical aspects of building large language models (LLMs). The speaker begins with an overview of key components (architecture, training, data, evaluation, systems) then delves into pre-training (classical language modeling) and post-training (AI assistant development). Specific attention is given to tokenization, evaluation metrics (perplexity and benchmarks like HELM and MMLU), and data challenges. The speaker also discusses scaling laws, showing how increased compute correlates with improved performance and how this informs resource allocation. Finally, the lecture covers supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF), highlighting the use of LLMs to improve data collection efficiency and the challenges of evaluating open-ended chatbot responses.

Outlines

Sign in to continue reading, translating and more.

Continue

Stanford Online

Introduction to LLMs and Key Training Components

Tokenization and its Importance in LLMs

Evaluating LLMs: Perplexity and Beyond

Data Collection and Preprocessing for LLMs

Scaling Laws in LLM Training

Optimizing Training Resources with Scaling Laws

Post-Training: Aligning LLMs for AI Assistants

Reinforcement Learning from Human Feedback (RLHF) and its Alternatives

Evaluating Post-Training and Alignment

System Optimization for LLM Training and Conclusion

Stanford CS229 I Machine Learning I Building Large Language Models (LLMs)

Stanford Online

00:05Introduction to LLMs and Key Training Components

Introduction to LLMs and Key Training Components

10:44Tokenization and its Importance in LLMs

Tokenization and its Importance in LLMs

20:05Evaluating LLMs: Perplexity and Beyond

Evaluating LLMs: Perplexity and Beyond

33:01Data Collection and Preprocessing for LLMs

Data Collection and Preprocessing for LLMs

42:20Scaling Laws in LLM Training

Scaling Laws in LLM Training

53:05Optimizing Training Resources with Scaling Laws

Optimizing Training Resources with Scaling Laws

1:00:06Post-Training: Aligning LLMs for AI Assistants

Post-Training: Aligning LLMs for AI Assistants

1:12:15Reinforcement Learning from Human Feedback (RLHF) and its Alternatives

Reinforcement Learning from Human Feedback (RLHF) and its Alternatives

1:22:53Evaluating Post-Training and Alignment

Evaluating Post-Training and Alignment

1:33:46System Optimization for LLM Training and Conclusion

System Optimization for LLM Training and Conclusion