Announcing On-Demand State Repartitioning for Apache Spark™ Structured Streaming on Databricks

Introduces On-Demand State Repartitioning for Spark Structured Streaming, enabling dynamic repartitioning of state without job restarts. This feature directly addresses the operational overhead and downtime associated with manual state management in long-running streaming applications, improving scalability and resource efficiency.