Building KVStore for ML Workloads at Pinterest | Jia Zhan | @Scale

Pinterest's KVStore, a distributed key-value store, is critical for machine learning feature serving, supporting key functions like home feed, related pins, search, and ad ranking. Jia Zhan, a software engineer at Pinterest, details KVStore's architecture, which accommodates both bulk data uploads from S3 and real-time streaming updates, using ROSDB as its storage engine and Apache Helix for shard management. Challenges include network throttling, addressed by fine-grained S3 rate limiting and traffic pacing, and noisy neighbor problems in the multi-tenant platform, mitigated by rate limiting, quota management, and priority-based load shedding. KVStore handles 170 million requests per second across 28,000 virtual machines, processing 5,000 petabytes of data daily and is evolving to meet demands for near real-time processing, higher throughput, and larger feature sets.

Outlines

Part 1: Introduction, Context

Part 2: Architecture, Data Ingestion

Part 3: Challenges, Optimizations

Part 4: Scale, Future Outlook

Sign in to continue reading, translating and more.

Continue

Building KVStore for ML Workloads at Pinterest | Jia Zhan

@Scale

Part 1: Introduction, Context

Pinterest's Machine Learning Infrastructure: The Role of ML Feature Serving

ML Feature Serving: Relevance Scoring and Key Requirements for Pinterest's KVStore

KVStore: Pinterest's Distributed Key-Value Store for ML Feature Serving

Part 2: Architecture, Data Ingestion

Data Ingestion Methods for KVStore: Batch, Incremental, and Streaming Updates

KVStore Internals: Read-Only and Mutable Modes for Data Updates

KVStore Data Model and API: Schema-less Flexibility for ML Features

Part 3: Challenges, Optimizations

Overcoming Network Throttling in KVStore: Visibility and Traffic Smoothing

Addressing Noisy Neighbor Problems in KVStore's Multi-Tenant Platform

Cost Efficiency in KVStore: Optimizations Across Compute, Storage, and Networking

Part 4: Scale, Future Outlook

KVStore Today: Powering Pinterest's ML Feature Serving at Scale

Future Trends and Directions for KVStore: Real-Time Processing and Scalability

Building KVStore for ML Workloads at Pinterest | Jia Zhan

@Scale

Part 1: Introduction, Context

00:01Pinterest's Machine Learning Infrastructure: The Role of ML Feature Serving

Pinterest's Machine Learning Infrastructure: The Role of ML Feature Serving

00:54ML Feature Serving: Relevance Scoring and Key Requirements for Pinterest's KVStore

ML Feature Serving: Relevance Scoring and Key Requirements for Pinterest's KVStore

02:16KVStore: Pinterest's Distributed Key-Value Store for ML Feature Serving

KVStore: Pinterest's Distributed Key-Value Store for ML Feature Serving

Part 2: Architecture, Data Ingestion

04:04Data Ingestion Methods for KVStore: Batch, Incremental, and Streaming Updates

Data Ingestion Methods for KVStore: Batch, Incremental, and Streaming Updates

05:26KVStore Internals: Read-Only and Mutable Modes for Data Updates

KVStore Internals: Read-Only and Mutable Modes for Data Updates

07:26KVStore Data Model and API: Schema-less Flexibility for ML Features

KVStore Data Model and API: Schema-less Flexibility for ML Features

Part 3: Challenges, Optimizations

08:53Overcoming Network Throttling in KVStore: Visibility and Traffic Smoothing

Overcoming Network Throttling in KVStore: Visibility and Traffic Smoothing

10:52Addressing Noisy Neighbor Problems in KVStore's Multi-Tenant Platform

Addressing Noisy Neighbor Problems in KVStore's Multi-Tenant Platform

12:35Cost Efficiency in KVStore: Optimizations Across Compute, Storage, and Networking

Cost Efficiency in KVStore: Optimizations Across Compute, Storage, and Networking

Part 4: Scale, Future Outlook

14:38KVStore Today: Powering Pinterest's ML Feature Serving at Scale

KVStore Today: Powering Pinterest's ML Feature Serving at Scale

16:31Future Trends and Directions for KVStore: Real-Time Processing and Scalability

Future Trends and Directions for KVStore: Real-Time Processing and Scalability