Enough with the Bad Benchmarks: Tools for Production-Grade Research - Qdrant

This post introduces Qdrant-FineWeb-10B, a 25 TiB open-source vector search benchmark dataset, and Supernova, an open-source distributed benchmarking framework. Supernova automates embedding generation, ground-truth calculation, database loading, and evaluation benchmarking, with modules like `nova-embed`, `nova-bf`, `nova-load`, and `nova-storm`. It integrates with SkyPilot for distributed compute across various cloud and HPC environments. Additional datasets, PubMed-Multi-Vector and Coyo-Vector-Embeddings, are also released.