Episode Summary

In this episode, Maribel Lopez interviews Kate Soule, Director of Technical Product Management for IBM's Granite products. They discuss IBM's third-generation AI models, their focus on efficiency and enterprise readiness, and the latest advancements including vision capabilities and reasoning features.

Guest

Kate Soule - Director of Technical Product Management for IBM's Granite products

Key Topics & Timestamps

00:04 - Introduction

Maribel introduces the show and Kate Soule
Brief overview of IBM Granite as fit-for-purpose, open-source enterprise AI models

00:48 - What is IBM Granite?

Designed as core building blocks for enterprises building with generative AI
Focus on efficiency with smaller model sizes
Monthly innovation updates to keep pace with rapidly evolving field

02:19 - Understanding AI Reasoning

Explanation of reasoning capabilities in AI models
How allowing models to generate more text at inference time can improve performance
Cost/benefit tradeoffs of reasoning features

03:13 - Enterprise AI Model Selection Criteria

Moving beyond "one model to rule them all" thinking
Importance of fit-for-purpose models
Why smaller models can be customized more easily
Trust and transparency considerations

05:38 - AI Governance and Safety

How to evaluate models for governance requirements
Safety evaluations and benchmarks as table stakes
Systems-based approach to safety with guardrails
IBM's Granite Guardian and protection mechanisms

08:55 - Benefits of Smaller Models

Why size matters: cost, latency, and customization advantages
Smaller models are easier to customize and require less computing power
IBM's transparent approach to training data

10:13 - Future of AI Evaluation

Performance per cost becoming the key evaluation metric
The growing importance of flexibility in model selection
How the "efficient frontier" between cost and performance will differentiate providers

12:41 - IBM's Vision Models

IBM's pragmatic enterprise focus for multimodal capabilities
Vision understanding (image in, text out) for practical business use cases
Specialization for documents, charts, and dashboards
Delivering powerful capabilities in only 2 billion parameters

15:25 - Understanding Model Size Context

Evolution from millions to billions of parameters
Practical considerations of deploying different-sized models
Finding the right cost-benefit trade-off for specific use cases

AI with Maribel Lopez: IBM Granite Models with Kate Soule

The AI with Maribel Lopez (AI with ML)