How to train your own Large Multimodal Model — with Hugo Laurençon & Leo Tronchon of HuggingFace M4 | Latent Space: The AI Engineer Podcast

This podcast episode delves into the multimodality in AI, particularly Hugging Face's research and contributions. Experts from Hugging Face discuss the company's research on multimodality, the development of tools like the Transformers library and the Datasets library, and the challenges and advancements in multimodal models such as Flamingo, Oblix, and IDFX.

Outlines

Sign in to continue reading, translating and more.

Continue

How to train your own Large Multimodal Model — with Hugo Laurençon & Leo Tronchon of HuggingFace M4

Latent Space: The AI Engineer Podcast

Hugging Face's Multimodality Research and the Future of AI

Exploring Multimodality: From CLIP and Vision Transformers to Flamingo and Oblix

Navigating the Challenges of Evaluating Multimodal Models: Benchmarks and Beyond

Exploring the Significance of Diverse Datasets in Multimodal Learning: Insights from Oblix

Exploring Data Diversity and Resolution Challenges in Multimodal Learning

Exploring Multimodal Data Sets: Nougat, MMC4, and Beyond

Oblix: Creating a High-Quality Multimodal Dataset

Exploring Multimodality and Synthetic Data in AI: Insights from IDFX Development

Troubleshooting Training Instabilities in Deep Learning Models: A Discussion on Techniques and Challenges

Exploring CLIP Variations and Improvements: A Journey Towards Better Image-Text Models

Edefix V2: Improving Image Resolution and Reducing Parameters

Multimodal Models and Hallucinations: Challenges and Evaluation

Multimodal Models: The Future of AI

Obelix: A Multitask Model for Image-to-HTML Generation

How to train your own Large Multimodal Model — with Hugo Laurençon & Leo Tronchon of HuggingFace M4

Latent Space: The AI Engineer Podcast

00:06Hugging Face's Multimodality Research and the Future of AI

Hugging Face's Multimodality Research and the Future of AI

05:45Exploring Multimodality: From CLIP and Vision Transformers to Flamingo and Oblix

Exploring Multimodality: From CLIP and Vision Transformers to Flamingo and Oblix

12:49Navigating the Challenges of Evaluating Multimodal Models: Benchmarks and Beyond

Navigating the Challenges of Evaluating Multimodal Models: Benchmarks and Beyond

16:40Exploring the Significance of Diverse Datasets in Multimodal Learning: Insights from Oblix

Exploring the Significance of Diverse Datasets in Multimodal Learning: Insights from Oblix

21:00Exploring Data Diversity and Resolution Challenges in Multimodal Learning

Exploring Data Diversity and Resolution Challenges in Multimodal Learning

26:44Exploring Multimodal Data Sets: Nougat, MMC4, and Beyond

Exploring Multimodal Data Sets: Nougat, MMC4, and Beyond

31:56Oblix: Creating a High-Quality Multimodal Dataset

Oblix: Creating a High-Quality Multimodal Dataset

36:42Exploring Multimodality and Synthetic Data in AI: Insights from IDFX Development

Exploring Multimodality and Synthetic Data in AI: Insights from IDFX Development

40:48Troubleshooting Training Instabilities in Deep Learning Models: A Discussion on Techniques and Challenges

Troubleshooting Training Instabilities in Deep Learning Models: A Discussion on Techniques and Challenges

44:15Exploring CLIP Variations and Improvements: A Journey Towards Better Image-Text Models

Exploring CLIP Variations and Improvements: A Journey Towards Better Image-Text Models

49:35Edefix V2: Improving Image Resolution and Reducing Parameters

Edefix V2: Improving Image Resolution and Reducing Parameters

54:57Multimodal Models and Hallucinations: Challenges and Evaluation

Multimodal Models and Hallucinations: Challenges and Evaluation

59:13Multimodal Models: The Future of AI

Multimodal Models: The Future of AI

1:05:11Obelix: A Multitask Model for Image-to-HTML Generation

Obelix: A Multitask Model for Image-to-HTML Generation