Pinterest Feature Trails

Feature trailActive

User Interest Clustering for Recommendation Systems

Pinterest evolved its user journey inference system by replacing a multi-stage clustering pipeline with a fine-tuned 4-billion-parameter LLM. This generative approach directly infers a ranked list of journey names from chronological activity logs, improving specificity, coherence, and handling of multilingual contexts. The system leverages supervised fine-tuning and LoRA on open-weight models, with the 4B parameter student offering the best balance of quality and throughput. Serving optimizations for batch processing at notification scale include PagedAttention and continuous batching.

8 posts · 2026

Timeline

Feature trailActive

Metrics Platform

Pinterest built Metrics Board, an end-to-end platform to automate metric creation, enforce quality, and enable agent-driven analytics. It unifies metric definitions in a Git repo, generates Airflow DAGs for computation, integrates rule-based and ML-based quality checks, and publishes metrics to various downstream systems like data catalogs, experimentation platforms, and BI tools. The system aims to improve metric discoverability, trust, and velocity for data producers and consumers, including AI agents.

1 post · 2026

Timeline

Feature trailActive

Automated Schema Evolution for DB Ingestion

Pinterest's next-generation database ingestion framework, built on Kafka, Flink, Spark, and Iceberg, has evolved to address data completeness challenges in stream-based processing. This includes automated schema evolution and a new partition finalization mechanism. The partition finalization logic, integrated into the Flink-to-Iceberg sink, uses event time statistics collected via Iceberg snapshot summaries and t-digest sketches. A watermark is generated to indicate partition completeness, stored as an Iceberg table property and propagated to downstream base tables. This mechanism is extensible through 'Accumulators' for metric collection and 'CommitProcessors' for custom post-commit logic, ensuring zero overhead when not configured.

3 posts · 2026

Timeline

Feature trailActive

Shopping Conversion Candidate Generation

Pinterest evolved its candidate generation for shopping conversion ads by introducing a 3-tower engagement co-train model. This model jointly predicts pCTR, pGCTR30, and pOCTR, leveraging a two-stage scoring flow that combines fast two-tower predictions with richer three-tower predictions for a selected subset of candidates. Key architectural components include a cross tower for user sequence and candidate feature interaction, and an inter module for shared representation learning. Serving optimizations, such as moving Pin pre-processing into the Pin tower and using lower precision, reduced inference latency significantly. The new model achieved substantial offline loss reduction for three-tower predictions and translated to online lifts in CTR and gCTR30, while maintaining two-tower prediction quality and reducing cost per click.

2 posts · 2026

Timeline

Feature trailActive

VLM Serving Stack

Pinterest built a VLM serving stack on NVIDIA Dynamo and Blackwell GPUs to handle multimodal workloads. This stack addresses challenges like complex request payloads, expensive prefill, multi-turn interactions with evolving visual context, and KV cache pressure. Key features include disaggregated prefill/decode serving, KV cache offloading (via LMCache), multimodal support in Dynamo frontend/vLLM, and custom modality support for projection embeddings (using PinCLIP) to reduce the cost of image processing. The system was further optimized by integrating custom modality support for projection embeddings using PinCLIP, allowing the reuse of precomputed embeddings and avoiding expensive pixel-based image serving.

2 posts · 2026

Timeline

Feature trailActive

AI-Assisted Engineering Operations

Pinterest is evolving its engineering operations by integrating 'AI Teams' as a fundamental shift in how teams operate, moving beyond simply using AI tools to a paradigm shift in ownership, strategic planning, and execution. This involves automating routine tasks, accelerating analysis, and rapid prototyping, freeing up cognitive resources for higher-level strategic thinking. The core value of an engineering team shifts from expert coders to expert strategists and problem-solvers, using AI as a tool. This post details the implementation of a testing process to optimize agent performance and skill invocation for AI coding assistants, providing metrics and optimization strategies.

4 posts · 2026

Timeline

Feature trailActive

Infrastructure as Code Security and Provisioning

Pinterest introduced the Resource Provisioner Pipeline (RPP), a specialized Terraform execution engine designed to safely manage infrastructure changes across a distributed, multi-repository architecture. RPP enforces compliance and security guardrails for AWS operations through dual controls, centralized GitHub Actions execution, and secure role-chaining, managing hundreds of Terraform workspaces and tens of thousands of resources including security policies, networking infrastructure, and storage. The system uses OIDC token validation for secure role assumption, a central source-of-truth configuration file for workspace property determination, and strict backend validation to ensure integrity. It also incorporates vulnerability scanning and pre-deployment local testing. The RPP provides unified auditing, enables instant systemic patches for vulnerabilities, enforces path-to-workspace mapping, tracks resource metrics, and sets the stage for automated tag enforcement.

2 posts · 2026

Timeline

Feature trailActive

Foundation Model Training Scalability

Pinterest has achieved near-linear training scalability for its foundation models by addressing communication bottlenecks in distributed training. Initial efforts focused on enabling AWS EFA for OS-bypass networking. Subsequent optimizations included Quantized Communications (QComms) to reduce payload size, Balanced Sharding to ensure even workload distribution, Bandwidth-Aware Embedding Optimization by reshaping embedding dimensions, and a refined 2D Parallel topology (All-to-All optimized) to improve communication efficiency. This has led to significant engagement gains across Pinterest's recommendation surfaces.

5 posts · 2026

Timeline

Feature trailActive

ML Workload Network Efficiency

Pinterest optimized its ML serving architecture by addressing network bottlenecks between root and leaf partitions. Initial efforts focused on enabling compression (LZ4) to reduce bandwidth usage. The core innovation, 'Feature Trimmer,' was introduced to send only necessary features by leveraging model signatures as the source of truth. This involved synchronizing model deployments with root configurations through staged delivery, significantly reducing network traffic and enabling infrastructure savings.

1 post · 2026

Timeline

Feature trailActive

URL Normalization for Content Deduplication

Pinterest developed the Minimal Important Query Param Set (MIQPS) algorithm to dynamically learn and apply URL normalization rules. This system automatically identifies and strips non-essential URL parameters, significantly reducing redundant content ingestion and processing. MIQPS complements static rules by adapting to the long tail of domains with varying URL structures, using empirical testing based on page content identity to distinguish important parameters from noise. An anomaly detection layer guards against regressions by comparing new MIQPS computations against previous versions, rejecting updates if too many patterns become anomalous.

1 post · 2026

Timeline