Departing Google: Transformer Paper Author's Sakana AI Adventure

This podcast episode discusses Sakana AI, a Japanese AI startup founded by former Google AI employees, focusing on a nature-inspired approach to AI model development. The speaker explains Sakana AI's concept of using many smaller, interconnected models instead of one large model, similar to a school of fish, to improve efficiency, reduce costs, and mitigate legal risks associated with massive datasets. The speaker highlights the company's location in Tokyo as a strategic move to diversify the AI industry beyond Silicon Valley and leverage diverse perspectives. The podcast concludes by suggesting that Sakana AI's approach could significantly disrupt the AI industry if successful, potentially influencing larger companies to adopt similar strategies. ChatGPT's segmented model structure is presented as a current example of a similar, albeit less granular, approach.