Our step-by-step guide to evaluating runtime security tools

This post details a rigorous evaluation process for runtime security tools, including attack simulations on Kubernetes clusters and Linux servers. It highlights the limitations of cloud audit logs and the necessity of runtime security tools for threat detection, incident response, and forensics. Key evaluation points included detection capability (built-in rules, custom rules, alert quality) and incident response support (log richness, log searchability). The evaluation process involved developing attack scenarios (exploiting GitLab vulnerabilities, compromising developer laptops), setting up parallel Kubernetes and VM environments, executing attacks, and analyzing results. Key learnings emphasize the importance of self-testing commercial products, collaborating with vendors for improvements, understanding tool boundaries, and the critical need for continuous runtime event logging in Kubernetes.