Open Sourcing Bullet, Yahoo’s Forward-Looking Query Engine for Streaming Data

This post announces the open-sourcing of Bullet, a forward-looking general purpose query engine for streaming data. It details Bullet's architecture, which operates on data flowing through the system after a query is submitted, eliminating the need for a persistence layer. This 'look-forward' approach allows for efficient querying of streaming data, scanning data only once per query. The system is multi-tenant, linearly scalable, and has been tested to handle high query volumes and data rates. It is currently implemented on Storm and can be extended to support other stream processing engines like Spark Streaming or Flink. The post also outlines its usage at Yahoo for validating application instrumentation and in continuous delivery pipelines.