
The current concentration of AI power within a few large corporations is a temporary byproduct of Transformer architecture rather than an inherent feature of artificial intelligence. Lukasz Kaiser, co-author of the "Attention Is All You Need" paper, explains that while today’s models require massive data centers and billions of dollars to scale, the next research breakthrough could enable high-performance models trained on significantly less data. Human intelligence serves as the primary proof of concept for this possibility, demonstrating that specialized, distributed expertise can outperform massive generalist systems. As major labs like OpenAI shift focus toward product development, a significant opportunity emerges for the open-source community and academia to lead fundamental research. Future advancements will likely move away from monolithic, "everything-at-once" models toward a diverse ecosystem of distributed, expert models that are more accessible and broadly empowering for individual users.
Sign in to continue reading, translating and more.
Open full episode in Podwise