Confluent

Sending Events to OpenSearch with the New Confluent Sink Connector

May 21, 2024Ahmed Saef Zamzam
Sending Events to OpenSearch with the New Confluent Sink Connector
SummaryAI, from the source post

Introduces a new fully managed OpenSearch Sink Connector for Confluent Cloud, simplifying the process of streaming data from Kafka topics to OpenSearch indexes. Details the connector's architecture, configuration options (1:1, M:1, 1:M, N:M topic-to-index mappings), scalability through connector tasks, and cost considerations. Addresses limitations and workarounds for private networking with OpenSearch (PrivateLink, Transit Gateway, VPC peering) and notes the current lack of support for batch inserts and OpenSearch Serverless authentication, with ongoing work to address these.

More from the Confluent Connectors Portfolio trail