Lightweight Feature Platform // Matt Bleifer & Mike Eastham // MLOps Podcast #209

The podcast explores the evolution of Tecton from a feature store to a comprehensive feature platform, highlighting the challenges and advancements in real-time AI. Matt Bleifer, product manager at Tecton, and Mike Eastham, chief architect, discuss the company's journey, including the shift from a Spark-centric architecture to the Python-native Rift compute engine. They address the complexities of feature engineering, orchestration, and monitoring, emphasizing the importance of decoupling features from code. The discussion also covers the integration of DuckDB for efficient data processing and the potential of LLMs in feature engineering, particularly in creating dynamic prompts for more context-aware AI applications.

Outlines

Part 1: Conference Announcements

Part 2: Tecton Introduction, Team, and Background

Part 3: Product Evolution and Feature Engineering

Part 4: Technical Architecture and Rift

Part 5: Performance and Industry Trends

Part 6: Design Principles and User Experience

Part 7: AI, LLMs, and Future Outlook

Part 8: Closing and Community

Sign in to continue reading, translating and more.

Open full episode in Podwise

MLOps.community

Part 1: Conference Announcements

MLOps Community Virtual Conference: February 15th and 22nd, Incredible Lineup

Keynote Speakers and AI Product Owners Track at MLOps Community Conference

Part 2: Tecton Introduction, Team, and Background

Tecton's Feature Platform: Decoupling Code from Feature Engineering

Tecton's Lightweight Product Evolution: Moving Beyond Spark Dependency

Introducing the Tecton Team: "Chief" Mike Eastham and "Big Dog" Matt Bleifer

Matt Bleifer's Journey: From Twitter's ML Platform to Tecton's Feature Store

Matt Bleifer's Reaction to Twitter's Rebrand to X: "That One Hurt Me Emotionally"

Mike Eastham's Role at Tecton: From First Engineer to Chief Architect

Part 3: Product Evolution and Feature Engineering

Reducing Complexity at Tecton: Harmonizing Functionality for Different Use Cases

Tecton's Evolution: From Feature Store to Feature Platform for Real-Time AI

Feature Store Misconceptions: Beyond Glorified Databases to Orchestration and Low Latency

Part 4: Technical Architecture and Rift

Tecton's Rift: A Python-Native Compute Engine for Simplified Feature Development

Tecton's Design Philosophy: Choosing the Right Tool for the Job, Avoiding Spark Overkill

Tecton's Use of DuckDB: A Great Experience for Feature Data Preparation

Rift's Architecture: Arrow-Based Data Transformation with Snowflake and DuckDB

Tecton's Stream Processing: Stateless Ingestion and Aggregation at Serving Time

Part 5: Performance and Industry Trends

Defining "Large" Aggregations: Balancing Performance and Scalability with Tecton

The "Big Data is Dead" Argument: Single-Node Processing for Most Use Cases

Simplifying the Product: Stream Architecture and Avoiding Over-Engineering

Part 6: Design Principles and User Experience

Tecton's Guiding Principles: Simplicity, Iteration Speed, and a Unified Product

Harmonious Coexistence: Spark and Rift Under One Roof for Diverse Teams

Balancing Simplicity and Flexibility: Tecton's Approach to Product Design

Part 7: AI, LLMs, and Future Outlook

Features vs. KPIs: Cross-Pollination and Data Sharing Between Systems

LLMs and Feature Stores: Productionizing RAG Systems and Dynamic Prompts

The Obviousness of LLMs and Feature Stores: Data to Model to Application

Tecton's Roadmap: LLM Investments, Rift Success, and Real-Time Decisioning Pipelines

Part 8: Closing and Community

Easier Tecton Trials and Demetrius's Conference Festival Aspirations

Surveys, Jam Sessions, and Fond Farewells

Lightweight Feature Platform // Matt Bleifer & Mike Eastham // MLOps Podcast #209

MLOps.community

Part 1: Conference Announcements

00:00MLOps Community Virtual Conference: February 15th and 22nd, Incredible Lineup

MLOps Community Virtual Conference: February 15th and 22nd, Incredible Lineup

00:26Keynote Speakers and AI Product Owners Track at MLOps Community Conference

Keynote Speakers and AI Product Owners Track at MLOps Community Conference

Part 2: Tecton Introduction, Team, and Background

02:10Tecton's Feature Platform: Decoupling Code from Feature Engineering

Tecton's Feature Platform: Decoupling Code from Feature Engineering

03:30Tecton's Lightweight Product Evolution: Moving Beyond Spark Dependency

Tecton's Lightweight Product Evolution: Moving Beyond Spark Dependency

04:30Introducing the Tecton Team: "Chief" Mike Eastham and "Big Dog" Matt Bleifer

Introducing the Tecton Team: "Chief" Mike Eastham and "Big Dog" Matt Bleifer

06:23Matt Bleifer's Journey: From Twitter's ML Platform to Tecton's Feature Store

Matt Bleifer's Journey: From Twitter's ML Platform to Tecton's Feature Store

08:30Matt Bleifer's Reaction to Twitter's Rebrand to X: "That One Hurt Me Emotionally"

Matt Bleifer's Reaction to Twitter's Rebrand to X: "That One Hurt Me Emotionally"

09:48Mike Eastham's Role at Tecton: From First Engineer to Chief Architect

Mike Eastham's Role at Tecton: From First Engineer to Chief Architect

Part 3: Product Evolution and Feature Engineering

11:30Reducing Complexity at Tecton: Harmonizing Functionality for Different Use Cases

Reducing Complexity at Tecton: Harmonizing Functionality for Different Use Cases

12:52Tecton's Evolution: From Feature Store to Feature Platform for Real-Time AI

Tecton's Evolution: From Feature Store to Feature Platform for Real-Time AI

18:22Feature Store Misconceptions: Beyond Glorified Databases to Orchestration and Low Latency

Feature Store Misconceptions: Beyond Glorified Databases to Orchestration and Low Latency

Part 4: Technical Architecture and Rift

20:55Tecton's Rift: A Python-Native Compute Engine for Simplified Feature Development

Tecton's Rift: A Python-Native Compute Engine for Simplified Feature Development

24:29Tecton's Design Philosophy: Choosing the Right Tool for the Job, Avoiding Spark Overkill

Tecton's Design Philosophy: Choosing the Right Tool for the Job, Avoiding Spark Overkill

26:11Tecton's Use of DuckDB: A Great Experience for Feature Data Preparation

Tecton's Use of DuckDB: A Great Experience for Feature Data Preparation

27:36Rift's Architecture: Arrow-Based Data Transformation with Snowflake and DuckDB

Rift's Architecture: Arrow-Based Data Transformation with Snowflake and DuckDB

29:46Tecton's Stream Processing: Stateless Ingestion and Aggregation at Serving Time

Tecton's Stream Processing: Stateless Ingestion and Aggregation at Serving Time

Part 5: Performance and Industry Trends

33:36Defining "Large" Aggregations: Balancing Performance and Scalability with Tecton

Defining "Large" Aggregations: Balancing Performance and Scalability with Tecton

37:05The "Big Data is Dead" Argument: Single-Node Processing for Most Use Cases

The "Big Data is Dead" Argument: Single-Node Processing for Most Use Cases

39:53Simplifying the Product: Stream Architecture and Avoiding Over-Engineering

Simplifying the Product: Stream Architecture and Avoiding Over-Engineering

Part 6: Design Principles and User Experience

41:13Tecton's Guiding Principles: Simplicity, Iteration Speed, and a Unified Product

Tecton's Guiding Principles: Simplicity, Iteration Speed, and a Unified Product

44:04Harmonious Coexistence: Spark and Rift Under One Roof for Diverse Teams

Harmonious Coexistence: Spark and Rift Under One Roof for Diverse Teams

45:53Balancing Simplicity and Flexibility: Tecton's Approach to Product Design

Balancing Simplicity and Flexibility: Tecton's Approach to Product Design

Part 7: AI, LLMs, and Future Outlook

47:08Features vs. KPIs: Cross-Pollination and Data Sharing Between Systems

Features vs. KPIs: Cross-Pollination and Data Sharing Between Systems

50:48LLMs and Feature Stores: Productionizing RAG Systems and Dynamic Prompts

LLMs and Feature Stores: Productionizing RAG Systems and Dynamic Prompts

56:02The Obviousness of LLMs and Feature Stores: Data to Model to Application

The Obviousness of LLMs and Feature Stores: Data to Model to Application

57:10Tecton's Roadmap: LLM Investments, Rift Success, and Real-Time Decisioning Pipelines

Tecton's Roadmap: LLM Investments, Rift Success, and Real-Time Decisioning Pipelines

Part 8: Closing and Community

59:02Easier Tecton Trials and Demetrius's Conference Festival Aspirations

Easier Tecton Trials and Demetrius's Conference Festival Aspirations

1:01:15Surveys, Jam Sessions, and Fond Farewells

Surveys, Jam Sessions, and Fond Farewells