Observability Pipeline Cost Optimization

3 posts · Mar 2023 – Jul 2026

What this trail covers

Elastic's Observability platform provides end-to-end monitoring for SAP workloads, integrating cloud infrastructure, SAP machine layer metrics (via Google Cloud's Agent for SAP and Elastic Agent), and SAP application layer data (via Kyndryl's custom connector). This unified approach leverages Kibana for visualization and AI-driven insights, aiming to reduce complexity and cost in monitoring SAP environments. The platform now includes enhanced SLO/SLI monitoring capabilities for defining, tracking, and visualizing service level objectives and indicators.

All Elastic trails

Jul 2026Elastic’s new metrics capabilities will dramatically improve uptime for public sector ITThis post details the engineering behind Elastic's new columnar metrics engine for its Observability platform. It highlights the re-architecture of Elasticsearch for time series data, resulting in a 6.6x reduction in storage footprint for OpenTelemetry metrics (down to 3.75 bytes/data point) and up to 160x faster time series queries compared to previous versions. It also emphasizes native Prometheus and PromQL support, allowing existing Prometheus tooling and queries to be used directly with Elastic, and introduces a migration platform for translating Grafana and Datadog assets. The pricing model is detailed, focusing on data volume rather than host count or cardinality, with metrics stored in TSDS index mode priced at 25% of the standard Observability per-GB rate.
Jun 2026How LivePerson cut observability pipeline costs by benchmarking Google Cloud infrastructureThis post details LivePerson's methodology for optimizing their observability pipeline costs on Google Cloud by benchmarking Logstash and Kafka performance across different machine types and compression codecs. Key findings include the superior cost-efficiency of n4d-standard-2 instances for Logstash and the performance gains from using LZ4 compression for Kafka. The post emphasizes the importance of infrastructure selection as a high-leverage cost optimization decision at scale and recommends periodic re-evaluation of cloud infrastructure choices.
Jul 2024Elastic's full-stack observability experience for your SAP workloadsThis post details the integration of Elastic Agent, Google Cloud's Agent for SAP, and Kyndryl's custom connector to provide full-stack observability for SAP workloads. It highlights how Elastic Agent collects logs, metrics, and traces from Google Cloud services and VMs, while Google Cloud's Agent for SAP specifically gathers SAP Host Agent metrics, process monitoring metrics, and workload manager validation metrics. Kyndryl's connector ingests SAP application layer data, including user activities, audit logs, and background job performance. The post emphasizes the unified data collection and visualization through Kibana for AI-driven insights and cost optimization.
Mar 2023An observability solution for today’s “always-on” worldThis post introduces Elastic's new SLO/SLI monitoring capability, allowing users to define custom SLOs (latency, availability, KQL-based), monitor their status, and set up alerts based on budget burn rates. It also highlights the cost efficiency benefits of Elastic's Time Series Database (TSDB) for downsampling and reducing storage costs of telemetry data. The post further details how Elastic Observability provides integrated full-stack views (e.g., APM traces with Kubernetes metrics/logs) and actionable insights with diagnostic information to reduce MTTR.
Advertisement