SE Radio 698: Srujana Merugu on How to build an LLM App

In this episode of Software Engineering Radio, Priyanka Raghavan interviews Srujana Merugu about building LLM-based applications. The discussion covers fundamental definitions such as generative AI, deep learning, pre-trained models, fine-tuning, and transformers. Srujana explains common use cases for generative AI, including chatbots, process automation, content creation, and scientific discovery, while also pointing out scenarios where LLMs might not be the best choice. The conversation extends to choosing the right model, key model metrics, and the architecture of LLM applications, emphasizing the importance of a modular, agentic system with components for planning, execution, memory, and validation. The episode concludes with insights on multi-sensory models and strategies for staying current with the rapid pace of innovation in the field.

Outlines

Part 1: Generative AI Fundamentals

Part 2: Use Cases and Model Selection

Part 3: Evaluation and Architecture

Part 4: Future Trends and Challenges

Sign in to continue reading, translating and more.

Continue

Software Engineering Radio - the podcast for professional software developers

Part 1: Generative AI Fundamentals

Introduction to Generative AI and Key Terminology

Pre-trained Models, Fine-tuning, and Transformers

Diffusion Models, Vision Transformers, and Multimodal Integration

Part 2: Use Cases and Model Selection

Common Use Cases of Generative AI

When Not to Use an LLM

Choosing the Right Model: E-commerce App Example

Solving for Different Capabilities: Understanding, Retrieval, and Generation

Part 3: Evaluation and Architecture

Model Metrics and Quantitative Evaluation

LLM Application Architecture: Agentic Approach

RAG Architecture and Validation

Prompting Techniques and Automated Prompt Optimization

Part 4: Future Trends and Challenges

Open Challenges, Multi-Sensory Models, and Keeping Up with Innovation

SE Radio 698: Srujana Merugu on How to build an LLM App

Software Engineering Radio - the podcast for professional software developers

Part 1: Generative AI Fundamentals

00:00Introduction to Generative AI and Key Terminology

Introduction to Generative AI and Key Terminology

04:38Pre-trained Models, Fine-tuning, and Transformers

Pre-trained Models, Fine-tuning, and Transformers

12:03Diffusion Models, Vision Transformers, and Multimodal Integration

Diffusion Models, Vision Transformers, and Multimodal Integration

Part 2: Use Cases and Model Selection

17:07Common Use Cases of Generative AI

Common Use Cases of Generative AI

23:15When Not to Use an LLM

When Not to Use an LLM

27:17Choosing the Right Model: E-commerce App Example

Choosing the Right Model: E-commerce App Example

35:09Solving for Different Capabilities: Understanding, Retrieval, and Generation

Solving for Different Capabilities: Understanding, Retrieval, and Generation

Part 3: Evaluation and Architecture

41:43Model Metrics and Quantitative Evaluation

Model Metrics and Quantitative Evaluation

48:56LLM Application Architecture: Agentic Approach

LLM Application Architecture: Agentic Approach

55:39RAG Architecture and Validation

RAG Architecture and Validation

1:03:15Prompting Techniques and Automated Prompt Optimization

Prompting Techniques and Automated Prompt Optimization

Part 4: Future Trends and Challenges

1:09:57Open Challenges, Multi-Sensory Models, and Keeping Up with Innovation

Open Challenges, Multi-Sensory Models, and Keeping Up with Innovation