Tri Dao: The End of Nvidia's Dominance, Why Inference Costs Fell & The Next 10X in Speed

In this episode of Unsupervised Learning, Jacob Effron interviews Tri Dao, a leading AI researcher and Chief Scientist at Together. They discuss the future of AI hardware, potential competitors to NVIDIA, and the stability of AI architectures for chip design. Tri shares his insights on optimizing models, the role of abstractions in hardware portability, and the use of AI in improving coding efficiency. The conversation also covers the advancements in AI inference, the diversification of workloads, and the importance of data in AI development, concluding with Tri's perspective on achieving expert-level AI and the future of robotics.

Outlines

Part 1: NVIDIA Ecosystem, AI Hardware

Part 2: Optimization Strategies

Part 3: Inference Optimization and Workloads

Part 4: Future AI and Research

Sign in to continue reading, translating and more.

Open full episode in Podwise

Unsupervised Learning: Redpoint's AI Podcast

Part 1: NVIDIA Ecosystem, AI Hardware

Introduction to Tri Dao's Work and the NVIDIA Ecosystem

Architecture Stability and the Future of AI Hardware

Part 2: Optimization Strategies

Strategic Bets for AI Startups and Optimizations in the NVIDIA Ecosystem

Abstractions for Cross-Chip Work and Hardware Portability

Domain-Specific Languages and AI-Assisted Kernel Development

Part 3: Inference Optimization and Workloads

AI's Role in Assisting Human Experts and Recent Improvements in AI Inference

Key Factors Driving Down Inference Costs and Improving Latency

Future Directions for Inference Optimization and Diversification of Use Cases

Workload Patterns and Optimization Strategies for Inference Providers

Evolution of the Inference Market and the Rise of Agentic Workloads

Part 4: Future AI and Research

Consumer Applications and the Pursuit of Expert-Level AI

Architectural Innovations and Research Directions

Academia vs. Industry and the Role of Exploration and Exploitation

Quick Fire Round and Closing Remarks

Tri Dao: The End of Nvidia's Dominance, Why Inference Costs Fell & The Next 10X in Speed

Unsupervised Learning: Redpoint's AI Podcast

Part 1: NVIDIA Ecosystem, AI Hardware

00:00Introduction to Tri Dao's Work and the NVIDIA Ecosystem

Introduction to Tri Dao's Work and the NVIDIA Ecosystem

04:00Architecture Stability and the Future of AI Hardware

Architecture Stability and the Future of AI Hardware

Part 2: Optimization Strategies

07:50Strategic Bets for AI Startups and Optimizations in the NVIDIA Ecosystem

Strategic Bets for AI Startups and Optimizations in the NVIDIA Ecosystem

11:59Abstractions for Cross-Chip Work and Hardware Portability

Abstractions for Cross-Chip Work and Hardware Portability

15:51Domain-Specific Languages and AI-Assisted Kernel Development

Domain-Specific Languages and AI-Assisted Kernel Development

Part 3: Inference Optimization and Workloads

21:08AI's Role in Assisting Human Experts and Recent Improvements in AI Inference

AI's Role in Assisting Human Experts and Recent Improvements in AI Inference

24:07Key Factors Driving Down Inference Costs and Improving Latency

Key Factors Driving Down Inference Costs and Improving Latency

29:18Future Directions for Inference Optimization and Diversification of Use Cases

Future Directions for Inference Optimization and Diversification of Use Cases

33:08Workload Patterns and Optimization Strategies for Inference Providers

Workload Patterns and Optimization Strategies for Inference Providers

37:30Evolution of the Inference Market and the Rise of Agentic Workloads

Evolution of the Inference Market and the Rise of Agentic Workloads

Part 4: Future AI and Research

42:30Consumer Applications and the Pursuit of Expert-Level AI

Consumer Applications and the Pursuit of Expert-Level AI

47:28Architectural Innovations and Research Directions

Architectural Innovations and Research Directions

52:07Academia vs. Industry and the Role of Exploration and Exploitation

Academia vs. Industry and the Role of Exploration and Exploitation

57:02Quick Fire Round and Closing Remarks

Quick Fire Round and Closing Remarks