Meta

Introducing Velox: An open source unified execution engine

Mar 9, 2023Pedro Pedreira, Masha Basmanova, Orri Erling
Introducing Velox: An open source unified execution engine
SummaryAI, from the source post

Introduced Velox, an open-source unified execution engine designed to accelerate and streamline the development of data management systems. Velox consolidates common data-intensive components of computation engines, offering a framework for consistent semantics and reducing work duplication. It integrates with existing systems like Presto (Prestissimo), Spark (Gluten), and PyTorch (TorchArrow), demonstrating significant performance improvements (up to 10x speedups) in various benchmarks and real-world workloads. Velox provides a generic type system, Arrow-compatible vector layout, vectorized expression evaluation, and common SQL operators, along with I/O and resource management primitives.

More from the Data Warehousing and Analytics Platform trail