Patricia Paskov, Director of Standards at AVERI, discusses the growing need for independent frontier AI auditing, the talent shortage in model evaluation, and pilot double-blind evaluations run with Google DeepMind and Openmind. Check out our sponsor Lovable: https://lovable.dev. Turn ideas into software people love.

MTS is a live news and interview show covering technology, business, politics, and culture as it happens.

Follow MTS:

X | ⁠https://twitter.com/MTSlive⁠

YouTube | ⁠https://www.youtube.com/@mtsituation⁠

Spotify | ⁠https://open.spotify.com/show/4HUNNmV1pjV7CW6Lphm1rM⁠

Apple | ⁠https://podcasts.apple.com/us/podcast/mts/id1891088763⁠

Substack | ⁠https://mtslive.substack.com/⁠

Interested in sponsoring the show? ⁠⁠⁠⁠⁠⁠sponsors@mts.now⁠⁠

--

Brought to you by:

Mongo DB - Anyone can prototype an agent. Learn how to get them into production on September 30th in New York - register at https://www.mongodb.com/events/mongodb-local/nyc

Lovable - Turn ideas into software people love with Lovable. Start today at https://lovable.dev/

Larridin - Finally prove what your AI investment is worth. Head to https://larridin.com/ today to start measuring your AI’s bottom line.

Neon (by Databricks) - Get up to $100K in credits for your startup at ⁠https://neon.com/mts⁠

ElevenLabs - AI that communicates at human level across every channel and modality. https://elevenlabs.io/mts

Arena - Measuring AI performance in the real world. ⁠https://arena.ai/⁠

Kong - The AI connectivity platform. Connect APIs, LLMs, agents, and systems with serious security and governance. https://konghq.com

Today’s stream is sponsored by VCX, the public ticker for private tech. Head over to http://getvcx.com to learn more

Blitzy - Autonomous software development for enterprise codebases. Ship 5x faster. https://blitzy.com

AdQuick - Make your brand a billboard. Out-of-home advertising as easy to scale as digital. https://adquick.com

Chargebee - Billing and monetization infrastructure for the AI era. https://www.chargebee.com/

--

Timestamps:

(00:00) Introduction

(00:53) What Does AVERI Do Right Now?

(02:23) How Many Qualified AI Auditors Exist?

(05:20) Addressing Conflict of Interest Concerns

(09:57) Gemini Double-Blind Evaluation Pilot

(12:38) How Do You Audit Model Alignment?

(15:54) Formal Verification and International Standards

--

Note: This podcast is not investment, legal, or tax advice, and is intended for informational and entertainment purposes only. Hosts and guests may hold positions in the companies and securities discussed; do your own research before acting on anything you hear.

How Double-Blind Tests Could Secure Frontier AI | Patricia Paskov

MTS

How Double-Blind Tests Could Secure Frontier AI | Patricia Paskov