”We’re Ahead of Where I Thought We’d Be” — Gemini 3 & the Future of AI

In this episode of The MAD Podcast, Matt Turck interviews Sebastian Bourgeaud, pre-training lead on Gemini 3 at Google DeepMind, about the architecture and development of Gemini 3, including the shift from a data unlimited regime to a data limited regime and the roles of pre-training and post-training. They discuss the progress of AI models, the organization of research teams at DeepMind, and the potential for AI in scientific discovery and automation of research. Bourgeaud shares his background and experiences, emphasizing the importance of research taste, managing complexity, and the integration of research efforts. The conversation covers the architecture of Gemini 3, the role of multimodal data, scaling laws, the use of synthetic data, and the challenges of evaluation and alignment in pre-training. They also touch on the future directions of AI research, including continual learning, long context capabilities, and the need for efficient and robust models.