Design a Streaming Percentile Estimator
Design a Streaming Percentile Estimator
You are receiving an unbounded stream of numeric observations and need to answer percentile queries without retaining every value. Design an estimator for a given quantile q (where 0 <= q <= 1) and explain precisely what an error statement of plus or minus epsilon (where 0 < epsilon < 1) guarantees.
Your design should address two primary targets:
- A deterministic single-stream summary whose returned value's rank is within
epsilon * Nof the target rank afterNobservations. - A centroid-based estimator, such as a t-digest, that prioritizes accuracy near the tails of the distribution.
Compare the two approaches in terms of update cost, memory usage, behavior on duplicate values and skewed distributions, practical merge behavior, and the strength of the guarantees each can honestly claim. Assume that a classic deterministic summary cannot be merged by simple concatenation or recompression while retaining its original error bound. Do not present empirical t-digest accuracy as a universal worst-case guarantee.
Constraints & Assumptions:
- Values arrive one at a time and may be repeated or highly skewed.
- Queries occur only after at least one observation.
- Exact minimum and maximum should be available for
q=0andq=1. - Rank error alone gives no numeric-value error bound without distribution assumptions.
- If distributed merging is proposed, state exactly which guarantee survives and under which invariant; otherwise, identify the limitation.
AI interview prep
Core dilemma
Balancing the strict, provable rank error guarantees of deterministic summaries against the practical, tail-focused accuracy of probabilistic/centroid-based methods, especially under unbounded streams and merge constraints.
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Design a Streaming Percentile Estimator
Design a Streaming Percentile Estimator
You are receiving an unbounded stream of numeric observations and need to answer percentile queries without retaining every value. Design an estimator for a given quantile q (where 0 <= q <= 1) and explain precisely what an error statement of plus or minus epsilon (where 0 < epsilon < 1) guarantees.
Your design should address two primary targets:
- A deterministic single-stream summary whose returned value's rank is within
epsilon * Nof the target rank afterNobservations. - A centroid-based estimator, such as a t-digest, that prioritizes accuracy near the tails of the distribution.
Compare the two approaches in terms of update cost, memory usage, behavior on duplicate values and skewed distributions, practical merge behavior, and the strength of the guarantees each can honestly claim. Assume that a classic deterministic summary cannot be merged by simple concatenation or recompression while retaining its original error bound. Do not present empirical t-digest accuracy as a universal worst-case guarantee.
Constraints & Assumptions:
- Values arrive one at a time and may be repeated or highly skewed.
- Queries occur only after at least one observation.
- Exact minimum and maximum should be available for
q=0andq=1. - Rank error alone gives no numeric-value error bound without distribution assumptions.
- If distributed merging is proposed, state exactly which guarantee survives and under which invariant; otherwise, identify the limitation.