Logstash Production Data Sampling to Pre-production
Safely sample production data into pre-production environments with Logstash

Safely sample production data into pre-production environments with Logstash

10/1/2024

What this post added

Introduces a new Logstash pattern for safely sampling production data into pre-production environments. This pattern utilizes UDP for pipeline-to-pipeline communication, enabling a subset of production events to be sent to a pre-production cluster without blocking the production pipeline. It contrasts this with the output isolator pattern, highlighting reduced operational risks and performance overhead by avoiding persistent queues. The post provides detailed Logstash pipeline configurations for both the production (common) and pre-production pipelines, demonstrating the use of a Ruby filter for sampling and UDP output.

Read the original post ↗