Generative AI 101: Tokens, Pre-training, Fine-tuning, Reasoning — With Dylan Patel

Generative AI operates by converting language into numerical tokens, which models process as vectors to predict subsequent patterns. Pre-training establishes a general understanding of world knowledge by minimizing prediction loss across vast datasets, while post-training—including reinforcement learning with human feedback—aligns these models with specific functional and safety objectives. Recent advancements in reasoning models allow systems to allocate additional compute to complex problems by generating extended thought processes before finalizing answers. Although algorithmic improvements and models like DeepSeek have significantly reduced inference costs, the industry continues to invest in massive data center infrastructure. This scaling is essential for unlocking advanced capabilities, such as automated software engineering, which require compute power far beyond what is needed for simple chat applications. Dylan Patel, founder of SemiAnalysis, provides this technical breakdown of the current AI landscape and the trajectory of future model development.