This podcast episode features an interview with Tengyu Ma, an assistant professor of computer science at Stanford and CEO of Voyage. Tengyu discusses his research agenda, including deep learning in various fields such as theory, RL, embeddings, optimizers, and reasoning tasks. He highlights key papers and work in his lab, including the development of a new optimizer called Sophia. The episode explores topics such as improving optimizers for large-language models, the ease of applying AI in industry environments, the concept and applications of Retrieval Augment Generation (RAG) systems, the cost implications of using long context transformers, the role of embedding models in managing long-term memory, agent chaining in RAG systems, and ways to enhance retrieval quality. The importance of embeddings models and vector-based search in RAG systems is emphasized, as well as the role of academia in AI innovation and efficiency improvement.
