Stanford CS229 Machine Learning | Spring 2026 | Lecture 16: Basic Concept in RL, Policy Gradient

Transformer architectures face significant memory and computational constraints, primarily due to the quadratic scaling of attention mechanisms and the high cost of storing KV caches. Group Query Attention mitigates these issues by sharing keys and values across multiple heads, while sliding window attention reduces computational complexity to linear time by limiting the attention span. Mixture of Experts (MoE) further optimizes performance by routing tokens to specialized sub-networks, enabling large parameter counts with sparse activation. Beyond architecture, large language models leverage in-context and zero-shot learning to adapt to new tasks without parameter updates. Supervised fine-tuning, specifically instruction tuning, reinforces these capabilities by training models on task-specific input-output pairs, allowing for efficient, lightweight customization across diverse domains without the need for full-scale retraining.