Introducing Tableflow: Unifying Streaming and Analytics

What this post added

Introduces Tableflow, a new capability that unifies streaming and analytical data by materializing Kafka topics into Apache Iceberg or Delta Lake tables. It leverages Kora Storage Layer innovations to write Kafka segments to parquet files and uses a metadata materializer that taps into Schema Registry for Iceberg metadata and Delta transaction logs, handling schema mapping, evolution, and type conversions. Tableflow also includes continuous compaction of small files and currently supports the Iceberg REST catalog, with plans for AWS Glue and Polaris Catalog integrations, and direct customer-owned object storage.