TechPrufer
Sign inGet started

Blogs›Datadog›Ruby Test Impact Analysis

Ruby Test Impact Analysis

Ruby Test Impact Analysis

1
posts
2024

Datadog developed a Ruby library for test impact analysis to significantly reduce testing time. Initially, existing solutions like Ruby's Coverage module and TracePoint were explored but found to have high performance overhead. The team then built a custom solution by diving into the Ruby VM and using interpreter events (RUBY_EVENT_LINE) via C extensions. This custom approach achieved a 60-80% performance overhead, compatible with other coverage tools, and offered flexibility for future enhancements. The library aims to seamlessly collect impacted source code files during test runs without requiring code changes from users, ensuring correctness and performance for CI pipelines.

2024

How we built a Ruby library that saves 50% in testing time | Datadog

11/1/2024

This post details the development of a Ruby library for test impact analysis. It covers the exploration of existing solutions (Ruby's Coverage module and TracePoint) and their limitations (performance overhead, compatibility issues). The core contribution is the creation of a custom solution by leveraging Ruby's C extension API to hook into interpreter events (RUBY_EVENT_LINE). This custom approach significantly reduced performance overhead (60-80%) compared to previous methods and was released as a native extension. The post also touches on initial optimizations to reduce string transformation overhead.