JSON Shredding in Milvus: 88.9x Faster JSON Filtering with Flexibility

What this post added

Introduces JSON Shredding, a new feature that optimizes the storage and querying of semi-structured JSON data in Milvus. It categorizes JSON keys into typed, dynamic, and shared, storing them in optimized columnar formats (Arrow/Parquet) or a compact binary-JSON column with an inverted index for shared keys. This enables schema-free agility with high query performance, demonstrated by benchmarks showing up to 88.9x speedup for shared key queries and 15-30x for typed key queries.