ML Robustness & Engineering - Andrew Ilyas (MIT) | Machine Learning Street Talk

This podcast episode dives into the intricacies of data modeling in machine learning, emphasizing its significance in understanding model predictions, evaluating data accuracy, and identifying important training examples. Ilyas outlines the evolution and challenges of data attribution methods like TRAQ for more efficient estimations, while also exploring the concepts of adversarial examples, data collection biases, and the ongoing debate surrounding the intersection of academia and industry. Ultimately, the discussion highlights the critical importance of robust model behavior, the nuanced relationship between data and reasoning in neural networks, and future directions for research that can bridge the gap between theoretical advancements and practical applications in machine learning.

Outlines

Sign in to continue reading, translating and more.

Continue

ML Robustness & Engineering - Andrew Ilyas (MIT)

Machine Learning Street Talk

The Machine Learning Pipeline: From Data Collection to Robustness

Data Modeling: Understanding the Relationship Between Data and Predictions

Evaluating Data Models: On-Distribution and Non-Random Data Sets

Data Model Embeddings: Universal Interpretability and Prediction

Future Directions for Data Modeling: Efficiency, Applications, and Generalization

TRAQ: A More Efficient Data Model Estimator

Evaluating Data Attribution Methods: The LDS Score

The Role of Random Projections in TRAQ

Data Attribution in Language Models: The F-Trace Dataset

Abstract Features and the Spectrum of Learning

Abstraction, Reasoning, and the Limits of Neural Networks

The Role of Data in the Emergence of Reasoning

Academia vs. Industry: The Trade-Off Between Scale and Focus

Adversarial Examples: Not Bugs, But Features

The Spectrum of Features: From Memorization to Robust Generalization

Mitigating Robustness Problems: Adversarial Training and Other Approaches

Inductive Bias and the Search for Robust Features

The Debate Over Adversarial Examples: Bugs or Features?

Black Box Attacks: Adversarial Attacks in Production Systems

Data Collection Bias: Understanding and Correcting for Bias in Data

Self-Selection Bias: A Theoretical Model of Bias in Data Collection

The Future of Machine Learning: Bridging the Gap Between Academia and Industry

ML Robustness & Engineering - Andrew Ilyas (MIT)

Machine Learning Street Talk

00:00The Machine Learning Pipeline: From Data Collection to Robustness

The Machine Learning Pipeline: From Data Collection to Robustness

06:31Data Modeling: Understanding the Relationship Between Data and Predictions

Data Modeling: Understanding the Relationship Between Data and Predictions

13:11Evaluating Data Models: On-Distribution and Non-Random Data Sets

Evaluating Data Models: On-Distribution and Non-Random Data Sets

17:47Data Model Embeddings: Universal Interpretability and Prediction

Data Model Embeddings: Universal Interpretability and Prediction

23:53Future Directions for Data Modeling: Efficiency, Applications, and Generalization

Future Directions for Data Modeling: Efficiency, Applications, and Generalization

26:29TRAQ: A More Efficient Data Model Estimator

TRAQ: A More Efficient Data Model Estimator

30:52Evaluating Data Attribution Methods: The LDS Score

Evaluating Data Attribution Methods: The LDS Score

35:34The Role of Random Projections in TRAQ

The Role of Random Projections in TRAQ

39:01Data Attribution in Language Models: The F-Trace Dataset

Data Attribution in Language Models: The F-Trace Dataset

41:27Abstract Features and the Spectrum of Learning

Abstract Features and the Spectrum of Learning

44:01Abstraction, Reasoning, and the Limits of Neural Networks

Abstraction, Reasoning, and the Limits of Neural Networks

48:03The Role of Data in the Emergence of Reasoning

The Role of Data in the Emergence of Reasoning

50:05Academia vs. Industry: The Trade-Off Between Scale and Focus

Academia vs. Industry: The Trade-Off Between Scale and Focus

52:50Adversarial Examples: Not Bugs, But Features

Adversarial Examples: Not Bugs, But Features

58:08The Spectrum of Features: From Memorization to Robust Generalization

The Spectrum of Features: From Memorization to Robust Generalization

1:00:27Mitigating Robustness Problems: Adversarial Training and Other Approaches

Mitigating Robustness Problems: Adversarial Training and Other Approaches

1:03:31Inductive Bias and the Search for Robust Features

Inductive Bias and the Search for Robust Features

1:08:57The Debate Over Adversarial Examples: Bugs or Features?

The Debate Over Adversarial Examples: Bugs or Features?

1:10:48Black Box Attacks: Adversarial Attacks in Production Systems

Black Box Attacks: Adversarial Attacks in Production Systems

1:15:37Data Collection Bias: Understanding and Correcting for Bias in Data

Data Collection Bias: Understanding and Correcting for Bias in Data

1:19:59Self-Selection Bias: A Theoretical Model of Bias in Data Collection

Self-Selection Bias: A Theoretical Model of Bias in Data Collection

1:23:49The Future of Machine Learning: Bridging the Gap Between Academia and Industry

The Future of Machine Learning: Bridging the Gap Between Academia and Industry