Apple Feature Trails

This post characterizes the performance of Diffusion Language Models (DLMs) against Autoregressive Language Models (ARMs). It analyzes trade-offs in arithmetic intensity, context scaling, batched inference, and sampling step reduction. The work highlights that while DLMs offer parallelism, they struggle with long contexts, and ARMs benefit more from batching. Opportunities for DLM acceleration are identified, particularly in reducing sampling steps for competitive latency.

This post introduces ARBITRAGE, a novel step-level speculative generation framework for Large Language Models (LLMs). ARBITRAGE dynamically routes generation based on the predicted advantage between draft and target models, using a lightweight router to determine when the target model is likely to produce a meaningfully better step. This approach approximates an ideal ORACLE, achieving near-optimal efficiency-accuracy trade-offs and reducing inference latency by up to 2x at matched accuracy on m. This work extends prior research by scaling Categorical Flow Maps (CFMs) to 1.7B parameters and 2.1T tokens, demonstrating competitive sample quality and near-data-level token entropy in as few as 4 inference steps. It also introduces a likelihood bound for CFMs in the semi-discrete setting and provides insights into challenges and solutions for training these models at scale, including loss weighting and time scheduling.

This post introduces novel applications of graph algorithms (PageRank, k-core decomposition, clustering coefficient) to the internal kNN graph generated by UMAP. These methods enhance data sensemaking by identifying representative data points, revealing data density, and detecting similar data neighborhoods, offering practical and competitive alternatives to existing methods.

This post introduces a memory-efficient audio synthesis architecture for on-device deployment, enabling real-time, high-fidelity speech generation within strict compute and memory constraints. The architecture decouples temporal and depth processing using a residual vector quantization (RVQ) representation with a streaming encoder, a temporal decoder, and a depth decoder. A single, reusable depth decoder with Diffusion Transformer (DiT)-style stage conditioning generates RVQ levels autoregressively. Causal sliding window attention with fixed-window key-value caching ensures constant memory complexity. Deployed on the Apple Matrix Coprocessor (AMX), it achieves ~10ms per generation step with low memory footprint and asset size, supporting long audio streams. This system is deployed in Siri Expressive Voices, improving Mean Opinion Score (MOS) for speech synthesis.