Build an LLM from Scratch 7: Instruction Finetuning

In this coding along video series, Sebastian Raschka, the author of "Build a Large Language Model from Scratch," guides viewers through teaching LLMs to follow instructions, focusing on building a small personal assistant similar to ChatGPT. He explains the process of fine-tuning a pre-trained model, emphasizing the importance of dataset preparation, including formatting data, tokenizing, and padding. The episode covers loading OpenAI weights, instruction fine-tuning, and evaluating the LLM's performance, highlighting the challenges in assessing free-form answers and introduces the use of OLAMA for automated evaluation. The discussion also extends to bonus materials like preference tuning with DPO and building a user interface for the fine-tuned model.

Outlines

Part 1: Introduction and Data Preparation

Part 2: Model Loading and Fine-Tuning

Part 3: Evaluation and Results

Sign in to continue reading, translating and more.

Continue

Sebastian Raschka

Part 1: Introduction and Data Preparation

Introduction to Instruction Fine-Tuning for LLMs

Preparing the Dataset for Supervised Instruction Fine-Tuning

Dataset Setup: Formatting, Tokenizing, and Padding

Implementing the Custom Collate Function

Preparing the Data Loader

Part 2: Model Loading and Fine-Tuning

Loading the Pre-trained LLM and Initial Performance Check

Instruction Fine-Tuning the LLM

Part 3: Evaluation and Results

Preparing the Model for Evaluation

Evaluating the Fine-Tuned LLM with OLAMA

Build an LLM from Scratch 7: Instruction Finetuning

Sebastian Raschka

Part 1: Introduction and Data Preparation

00:02Introduction to Instruction Fine-Tuning for LLMs

Introduction to Instruction Fine-Tuning for LLMs

05:03Preparing the Dataset for Supervised Instruction Fine-Tuning

Preparing the Dataset for Supervised Instruction Fine-Tuning

15:50Dataset Setup: Formatting, Tokenizing, and Padding

Dataset Setup: Formatting, Tokenizing, and Padding

26:06Implementing the Custom Collate Function

Implementing the Custom Collate Function

39:28Preparing the Data Loader

Preparing the Data Loader

Part 2: Model Loading and Fine-Tuning

46:54Loading the Pre-trained LLM and Initial Performance Check

Loading the Pre-trained LLM and Initial Performance Check

54:35Instruction Fine-Tuning the LLM

Instruction Fine-Tuning the LLM

Part 3: Evaluation and Results

1:13:54Preparing the Model for Evaluation

Preparing the Model for Evaluation

1:24:07Evaluating the Fine-Tuned LLM with OLAMA

Evaluating the Fine-Tuned LLM with OLAMA