Cloud Security Monitoring and Alerting Pipeline

Datadog has built a serverless security monitoring and alerting pipeline for its AWS cloud operations. This pipeline integrates with Slack, Duo, and PagerDuty to notify, alert, and authenticate potential security-relevant API calls. It centralizes data from multiple AWS accounts into a dedicated security account, using CloudWatch Event Rules, SNS, and SQS for cross-account data collection. A Lambda function triggers every two minutes to process API call data, which is then fed into a security orchestration framework (Komand) for logic processing, decision-making, and user notifications. The system categorizes API calls by severity (log, notify, alert) and uses interactive Slack messages with Duo authentication for user verification. All workflow details are logged to Elasticsearch for visualization and analysis.