CS 194/294-280 (Advanced LLM Agents) - Lecture 1, Xinyun Chen

Advanced Large Language Model Agents centers on optimizing reasoning performance through inference-time compute scaling. Reasoning models leverage increased token budgets to perform complex planning, decomposition, and trial-and-error processes, moving beyond simple prompt-response patterns. Key methodologies include Chain of Thought prompting, which allows models to adapt reasoning depth to task difficulty, and search-based selection techniques like self-consistency and tree-of-thought, which explore multiple solution branches to improve accuracy. While iterative self-improvement and reflection show promise in code generation and agentic workflows, their effectiveness remains constrained by the model's ability to accurately evaluate its own output without external oracle feedback. Ultimately, scaling reasoning performance requires balancing parallel exploration and sequential refinement, tailored to the specific capabilities of the underlying model and the complexity of the task at hand.