# Secops-es-benchmark: an open benchmark for AI security agents on real Elasticsearch telemetry

**URL:** <https://discuss.elastic.co/t/secops-es-benchmark-an-open-benchmark-for-ai-security-agents-on-real-elasticsearch-telemetry/388910>\
**Category:** Elastic Security\
**Created:** [July 30, 2026, 4:13pm UTC](https://discuss.elastic.co/t/secops-es-benchmark-an-open-benchmark-for-ai-security-agents-on-real-elasticsearch-telemetry/388910 "2026-07-30T16:13:07Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![azmatjan139](https://avatars.discourse-cdn.com/v4/letter/a/f17d59/32.png) [@azmatjan139](https://discuss.elastic.co/u/azmatjan139)\
**Post date:** [July 30, 2026, 4:13pm UTC](https://discuss.elastic.co/t/secops-es-benchmark-an-open-benchmark-for-ai-security-agents-on-real-elasticsearch-telemetry/388910/1 "2026-07-30T16:13:07Z")

</div>

# secops-es-benchmark: an open benchmark for AI agents that investigate breaches in Elasticsearch

_Real, labeled attack telemetry in Elasticsearch — measure your own agent against ground truth._

Today I'm open-sourcing **secops-es-benchmark** — the first open benchmark for SecOps investigation agents built natively on Elasticsearch and ECS.

It closes a real gap. A huge share of the world's security telemetry lives in Elasticsearch, yet there's been no public, labeled way to measure whether an AI agent can actually investigate in that environment. The datasets out there are either synthetic-and-labeled or real-but-unlabeled — neither lets you score an agent on the real task. This one does.

**What you get:**

- **~239k ECS documents** across 12+ Elastic data streams — Elastic Endpoint, Zeek, nginx, Suricata, and detection-engine alerts.
- **5 real, non-destructive intrusions** forming one end-to-end kill chain, mapped to **27 ATT&CK techniques** — layered on genuine production noise, so over-alerting gets punished.
- **A two-tier exam:** 54 deterministically-graded questions (precise extraction) plus 5 open-ended investigations scored by an evidence-grounded LLM judge.
- **A one-command runner** and a **live, read-only Elasticsearch + Kibana demo** — point your agent at it and get a score with zero setup.

You hand an agent a single alert. It writes ES|QL, pivots across the data, reconstructs the intrusion, and recommends a proportionate response — then it's scored against ground truth.

Works with Claude or any OpenAI-compatible model. Open source, reproducible, and ready to run against your own agent today.

👉 [GitHub - TocharianOU/secops-es-benchmark · GitHub](https://github.com/TocharianOU/secops-es-benchmark)