In this episode of Training Data, Bob McGrew, former Chief Research Officer at OpenAI, discusses the current state and future of AI development, focusing on the trifecta of pre-training, post-training, and reasoning. McGrew suggests that 2025 will be the year of reasoning, highlighting its potential for significant advancements. While pre-training faces diminishing returns, it remains crucial for improving architectures. Post-training is about shaping model personality, requiring human insight. McGrew controversially posits that the fundamental concepts needed for AGI may already be discovered. He also predicts that agents will be priced at the cost of compute, impacting traditional economic models, and that proprietary data will become less valuable. McGrew identifies robotics as a promising area for startups, enabled by recent advancements in LLMs and vision encoders. He also shares insights on managing research organizations, emphasizing the importance of caring for and understanding the needs of talented individuals.
