Detection Pipeline Lab
A full detection pipeline on a 7-day sample estate: nine log sources, 13 Sigma detections with correlation, CI tests, precision and time to detect, containment plans, telemetry blind spots, shift handoff and a replay of public attack data. Edit a rule and watch the alerts, tests and precision change.
Harbourline Forwarding is a fictional freight forwarder. Every person, shipment, payee and network in the stream is invented. Nothing leaves this tab.
Loading the stream and the rule engine.
Detection Pipeline Lab FAQ
What is in the sample stream?
Seven days of Okta System Log, AWS CloudTrail, Kubernetes audit, Microsoft 365 unified audit, OCSF endpoint process records and four application systems for Harbourline Forwarding, a fictional freight forwarder. Four attack chains and a set of benign look-alikes are seeded with labels written in advance, so every alert can be scored as a true or false positive. IP addresses come from the documentation ranges.
How are the detections run?
Each detection is Sigma 2.0 YAML. Base rules are matched by sigma.js, the engine behind the pwnsy Detection Workbench, after an enrichment step that joins helpdesk tickets, the change calendar, on-call incidents and a 30-day baseline. Multi-step detections use Sigma correlation rules (event_count, value_count and temporal_ordered), evaluated by this page's JavaScript.
What does the CI tab check?
Every detection carries positive and negative test events. CI compiles each rule, lints it, checks its ATT&CK technique against the v19.2 snapshot and runs every test. The same 13 rules and 46 tests run on every push in the public GitHub repository edwardtay/detections-as-code-logistics, which also parses each rule with pySigma.
How are precision and time to detect measured?
Alerts from the stream are labelled from the seeded ground truth and passed through the pwnsy Alert Budget engine. Precision is TP / (TP + FP) per rule with a Wilson 95% range. Time to detect runs from the first malicious event to the alert, including source lag and the rule schedule.
What is the public attack data replay?
Seven small samples from splunk/attack_data (Apache-2.0), copied exactly as published, run through the same pipeline: CloudTrail StopLogging and CreateAccessKey, S3 GetObject, Okta push fatigue, password spray and factor deactivation, and Exchange forwarding rules. Each row links to its source file.
Is anything uploaded?
No. The stream is generated in your browser, every rule runs in JavaScript on this page, and downloads are built locally. Nothing is sent to a server.
ATT&CK IDs come from the pinned MITRE ATT&CK v19.2 snapshot and OCSF classes from the OCSF 1.9.0 snapshot used across the detection suite. Replay samples are from splunk/attack_data under the Apache License 2.0, copied unchanged with a link to each source file. All other records are generated and fictional.