AI Engineer

Five hard earned lessons about Evals — Ankur Goyal, Braintrust

Perceptual Evaluations: Evals for Aesthetics — Diego Rodriguez, Krea.ai

How BlackRock Builds Custom Knowledge Apps at Scale — Vaibhav Page & Infant Vasanth, BlackRock

Form factors for your new AI coworkers — Craig Wattrus, Flatfile

Fuzzing in the GenAI Era — Leonard Tang, Haize Labs

Multi Agent AI and Network Knowledge Graphs for Change — Ola Mabadeje, Cisco

Ola Mabadeje from Cisco's Outshift group discusses how AI can be used to reduce failures in production during change management. Outshift uses an incubation phase to understand customer problems, create prototypes, and deliver minimum viable products. Ola highlights the challenges of representing real-world networks wi...

Wisdom-Driven Knowledge Augmented Generation at Scale - Chin Keong Lam, Patho AI

The Next Unicorns: 7 Top AI startups from the HF0 Residency

#define AI Engineer - Greg Brockman, OpenAI (ft. Jensen Huang)

This podcast episode features an interview with Greg Brockman, who discusses his journey into coding, starting from his initial interest in mathematics to his fascination with the tangible impact of programming. He recounts his early experiences at Stripe, emphasizing the challenges and the intense customer focus that ...

The Future of Evals - Ankur Goyal, Braintrust

Ankur Goyal from Braintrust discusses the current state and future of evaluations (evals) in AI product development. He highlights that while Braintrust customers run thousands of evals daily, the process remains largely manual. Goyal then introduces "Loop," an AI agent built into Braintrust, which automates the optimi...

Designing AI-Intensive Applications - swyx

How to look at your data — Jeff Huber (Chroma) + Jason Liu (567)

Jeff Huber and Jason Liu present a two-part session on how to effectively analyze data to improve AI systems. Jeff discusses the importance of using fast evals—query and document pairs—to quickly and inexpensively measure retrieval system performance and suggests using LLMs to generate realistic queries. He shares a ca...

On Engineering AI Systems that Endure The Bitter Lesson - Omar Khattab, DSPy & Databricks

Evals Are Not Unit Tests — Ido Pesok, Vercel v0

Ido Pesok, an engineer at Vercel working on V0, introduces evals at the application layer, distinguishing them from model-layer evals. He uses the analogy of a fruit letter counter app to illustrate the unreliability of LLMs and the importance of building reliable AI applications. Pesok emphasizes understanding the "co...

2025 is the Year of Evals! Just like 2024, and 2023, and … — John Dickerson, CEO Mozilla AI

John Dickerson discusses the evolution and future of AI evaluation, particularly focusing on why 2025 is poised to be the "year of the evals." He explains how AI monitoring and evaluation are interconnected and highlights the impact of ChatGPT's launch and subsequent budget shifts in enterprises. Dickerson points out t...

Vibe Coding with Confidence — Itamar Friedman, Qodo

Itamar Friedman, CEO and co-founder of Qodo, discusses "Vibe Coding" and the role of CLI (Command Line Interface) in enhancing developer confidence. He introduces Qodo's new CLI tool and outlines an agenda covering the evolution of AI in software development from autocomplete to multi-agent workflows. Friedman differen...

AI Automation that actually works: $100M, messy data, zero surprises - Tanmai Gopal, Hasura/PromptQL

Full Workshop: Realtime Voice AI — Mark Backman, Daily

Vision AI in 2025 — Peter Robicheaux, Roboflow

Practical tactics to build reliable AI apps — Dmitry Kuchin, Multinear