The Hidden Flaws in AI Safety & Evaluation Benchmarks | Prof. Jackie Chi Kit Cheung | Jay Shah

The podcast explores the reliability, pragmatics, and evaluation of large language models with Dr. Jackie Cheung, Associate Professor at McGill University. Cheung discusses his early interest in linguistics and how computational methods offer systematic approaches to language analysis. He shares his rationale for choosing academia over industry, emphasizing ownership, mentorship, and research freedom. The conversation covers the challenges of using LLMs as judges, highlighting issues with validity and the need for better definitions and context-specific evaluations. Cheung also addresses the importance of language revitalization efforts to ensure diverse voices shape AI technology. The podcast further examines the limitations of scaling LLMs and the need for efficiency and safety, as well as the risks of chasing trends versus focusing on fundamental research.

Outlines

Part 1: Background and Academic Journey

Part 2: Language Revitalization and Qualitative Research

Part 3: Rethinking LLM Evaluation and Benchmarking

Part 4: General Purpose vs. Specific Use Cases

Part 5: Safety, Bias, and Defining Harm

Part 6: Model Behavior, Hallucinations, and Summarization

Part 7: Future Outlook and Career Advice

Sign in to continue reading, translating and more.

Continue

The Hidden Flaws in AI Safety & Evaluation Benchmarks | Prof. Jackie Chi Kit Cheung

Jay Shah

Part 1: Background and Academic Journey

00:07The End of Scaling Era for Large Language Models

The End of Scaling Era for Large Language Models

00:58Jackie Cheung's Background: Sparked by Multilingualism and Computational Linguistics

Jackie Cheung's Background: Sparked by Multilingualism and Computational Linguistics

02:36The Appeal of AI Algorithms: Systematic Thinking and Data Processing in Linguistics

The Appeal of AI Algorithms: Systematic Thinking and Data Processing in Linguistics

04:48Choosing Academia: Ownership, Mentorship, and Research Freedom

Choosing Academia: Ownership, Mentorship, and Research Freedom

06:55Diversity in Research: Industry vs. Academia Incentives and Evaluation

Diversity in Research: Industry vs. Academia Incentives and Evaluation

Part 2: Language Revitalization and Qualitative Research

08:35Language Revitalization: Broadening Voices in AI Development

Language Revitalization: Broadening Voices in AI Development

12:25Data Sovereignty and Community Buy-In: Key Aspects of Language Revitalization

Data Sovereignty and Community Buy-In: Key Aspects of Language Revitalization

14:15Qualitative Work: Stepping Outside the Computer Science Comfort Zone

Qualitative Work: Stepping Outside the Computer Science Comfort Zone

Part 3: Rethinking LLM Evaluation and Benchmarking

15:53Questioning LLMs as Judges: A Premature and Problematic Approach

Questioning LLMs as Judges: A Premature and Problematic Approach

18:39Validity Concerns: Human Annotations and Sensitivity to Minor Changes

Validity Concerns: Human Annotations and Sensitivity to Minor Changes

21:31Defining Evaluation Metrics: The Importance of Good Definitions

Defining Evaluation Metrics: The Importance of Good Definitions

22:46Best Practices for Evaluation: Documentation, Definitions, and Context

Best Practices for Evaluation: Documentation, Definitions, and Context

25:11Evidence Centered Benchmark Design (ECBD): A Framework for Better Benchmarks

Evidence Centered Benchmark Design (ECBD): A Framework for Better Benchmarks

27:32ECBD Framework: Components and Justifications for Benchmark Design

ECBD Framework: Components and Justifications for Benchmark Design

29:54Reusing Benchmarks: Adapting ECBD for Specific Research Questions

Reusing Benchmarks: Adapting ECBD for Specific Research Questions

Part 4: General Purpose vs. Specific Use Cases

31:59The Gap in General Purpose Benchmarks: Diversity and Downstream Use Cases

The Gap in General Purpose Benchmarks: Diversity and Downstream Use Cases

34:00Multi-Purpose Models: Validating for Specific Downstream Use Cases

Multi-Purpose Models: Validating for Specific Downstream Use Cases

35:24Intelligence vs. Reasoning: Overloaded Terms and Contextual Interpretation

Intelligence vs. Reasoning: Overloaded Terms and Contextual Interpretation

38:06Mechanisms vs. Abstraction: Separating Prompting from Fundamental Understanding

Mechanisms vs. Abstraction: Separating Prompting from Fundamental Understanding

Part 5: Safety, Bias, and Defining Harm

40:15Safety as an Undefined Bundle: Auditing Helpfulness and Harmlessness Data Sets

Safety as an Undefined Bundle: Auditing Helpfulness and Harmlessness Data Sets

43:50Unanticipated Behaviors: Exaggerated Safety and Quality of Service Harm

Unanticipated Behaviors: Exaggerated Safety and Quality of Service Harm

47:17Patching vs. Fundamental Work: Addressing Bias in Large Language Models

Patching vs. Fundamental Work: Addressing Bias in Large Language Models

49:24Defining Harm: A Moving Target at the Intersection of Architecture and Social Engineering

Defining Harm: A Moving Target at the Intersection of Architecture and Social Engineering

Part 6: Model Behavior, Hallucinations, and Summarization

51:45Stochastic Chameleons: Generalization and Abstract Classes in LLMs

Stochastic Chameleons: Generalization and Abstract Classes in LLMs

53:41Probing LLMs: Irrelevant Contexts and Systematic Hallucinations

Probing LLMs: Irrelevant Contexts and Systematic Hallucinations

57:19Extrapolation and Generalization: The Limitations of LLMs

Extrapolation and Generalization: The Limitations of LLMs

1:00:36Regression to the Mean: The Limits of LLMs in Novel Situations

Regression to the Mean: The Limits of LLMs in Novel Situations

1:02:24Agentic Frameworks: Tools for Novel Synthesis with Critical Thinking

Agentic Frameworks: Tools for Novel Synthesis with Critical Thinking

1:04:52Faithfulness in Summarization: Post-Editing and Separating Concerns

Faithfulness in Summarization: Post-Editing and Separating Concerns

1:07:03Hallucinations in Summarization: Factual vs. Non-Factual Distinctions

Hallucinations in Summarization: Factual vs. Non-Factual Distinctions

1:08:50Training Strategies: Data, Modeling, and Distinguishing Possible Worlds

Training Strategies: Data, Modeling, and Distinguishing Possible Worlds

Part 7: Future Outlook and Career Advice

1:11:43The Future of Scaling: Diminishing Returns and Targeted Interventions

The Future of Scaling: Diminishing Returns and Targeted Interventions

1:13:41Efficiency and Safety: New Priorities Beyond Scaling

Efficiency and Safety: New Priorities Beyond Scaling

1:15:05Hallucinations and Reasoning: Evaluation Issues and Subtle Errors

Hallucinations and Reasoning: Evaluation Issues and Subtle Errors

1:16:32Getting Started in AI: Hands-On Experience and Interdisciplinary Knowledge

Getting Started in AI: Hands-On Experience and Interdisciplinary Knowledge

1:18:49Filtering Noise: Building a Solid Foundation in the Field