Building Agents at Scale: Lessons from the Front Lines With Gary Stafford

In this episode of AI Explained, host Will Pong interviews Gary Stafford, Principal Solutions Architect at AWS Trans, about building AI agents at scale. They discuss how enterprises should decide when to use an agentic approach, the importance of understanding the problem before applying AI, and the differences between Gen-AI and agentic systems. Gary shares use cases for agentic technology, including code development, back-office automation, and customer-facing products. The conversation covers the challenges and benefits of multi-agent systems, the role of standards like MCP and open telemetry, and the importance of testing and observability in agentic systems. Gary also provides advice on scaling AI solutions and ensuring safety and compliance.

Outlines

Part 1: Introduction and Foundations

Part 2: Business Applications and Benefits

Part 3: Observability, Management, and Development

Part 4: Scaling, Security, and Safety

Part 5: Conclusion

Sign in to continue reading, translating and more.

Continue

AI Explained

Part 1: Introduction and Foundations

00:06Introduction to Building AI Agents at Scale

Introduction to Building AI Agents at Scale

01:34Determining the Right Approach: AI Agents vs. Traditional Methods

Determining the Right Approach: AI Agents vs. Traditional Methods

02:56Enterprise AI Initiatives: Top-Down Mandates vs. Process Transformation

Enterprise AI Initiatives: Top-Down Mandates vs. Process Transformation

04:48Use Cases for Agentic Technology and Distinguishing Gen-AI from Agents

Use Cases for Agentic Technology and Distinguishing Gen-AI from Agents

06:22Core Components of AI Agents and Deterministic vs. Non-Deterministic Decision-Making

Core Components of AI Agents and Deterministic vs. Non-Deterministic Decision-Making

Part 2: Business Applications and Benefits

08:15Business Use Cases for Agentic Approaches

Business Use Cases for Agentic Approaches

09:54The Role of Institutional Knowledge in Back-Office Automation

The Role of Institutional Knowledge in Back-Office Automation

11:22Automating the Sales Process with AI Agents

Automating the Sales Process with AI Agents

13:03Benefits of AI Agents in Enterprise Processes

Benefits of AI Agents in Enterprise Processes

14:57Leveraging AI to Better Serve Customers

Leveraging AI to Better Serve Customers

16:26MCP Servers and the Importance of Standards

MCP Servers and the Importance of Standards

18:31Multi-Agent Systems and Agent-to-Agent Communication

Multi-Agent Systems and Agent-to-Agent Communication

19:31Translating Human Structures into Agent Architectures

Translating Human Structures into Agent Architectures

20:38Transforming Processes with Agent-to-Agent Communication

Transforming Processes with Agent-to-Agent Communication

22:19Tools for Agent-to-Agent Conversation

Tools for Agent-to-Agent Conversation

Part 3: Observability, Management, and Development

23:54Observability and Open Telemetry (OTEL)

Observability and Open Telemetry (OTEL)

25:07Real-Time Logging and Observability in Agentic Systems

Real-Time Logging and Observability in Agentic Systems

26:49Managing and Tuning Agents

Managing and Tuning Agents

28:05Building a Recommendation Engine with an Agentic Approach

Building a Recommendation Engine with an Agentic Approach

29:51Challenges and Patterns in Agentic System Development

Challenges and Patterns in Agentic System Development

31:23Testing and Tuning Agentic Systems

Testing and Tuning Agentic Systems

32:47Developing Non-Deterministic Tests

Developing Non-Deterministic Tests

Part 4: Scaling, Security, and Safety

34:46Pitfalls in Scaling Agentic Solutions

Pitfalls in Scaling Agentic Solutions

36:51Identity, Access, and Security in Production

Identity, Access, and Security in Production

39:06Agent Management and Deployment at Scale

Agent Management and Deployment at Scale

40:10The Tension Between Ease of Use and Software Engineering Techniques

The Tension Between Ease of Use and Software Engineering Techniques

42:30The Importance of Domain Expertise

The Importance of Domain Expertise

44:44Ensuring Safety and Compliance at Scale

Ensuring Safety and Compliance at Scale

45:56Testing and Model Evolution

Testing and Model Evolution

47:33Adapting Software Development Processes

Adapting Software Development Processes

49:04Rapid Iteration and Non-Deterministic Testing

Rapid Iteration and Non-Deterministic Testing

50:46Preventative Medicine and Runtime Protection

Preventative Medicine and Runtime Protection

52:55Guardrails as a Starting Point

Guardrails as a Starting Point

54:00Adding an Orchestrator vs. Extending a Single Agent

Adding an Orchestrator vs. Extending a Single Agent

55:11Challenges in Connecting Multiple Agents

Challenges in Connecting Multiple Agents

56:06Model Specialization and Agent Separation

Model Specialization and Agent Separation

Part 5: Conclusion

57:51Conclusion and Thank You