Deploying Kafka Streams and KSQL with Gradle – Part 2: Managing KSQL Implementations | Confluent

What this post added

This post details the challenges of managing KSQL dependencies and pipelines in a source code repository. It introduces a custom Gradle plugin, `gradle-confluent`, which uses a file and directory structure to declare dependencies between KSQL scripts. The plugin ensures scripts are executed in the correct order, addressing issues with recreating KSQL streams and tables by managing dependencies between pipelines and within pipelines. It also highlights the complexity of dropping and recreating KSQL objects due to persistent query dependencies and the need for a more robust dependency management solution.