# How do I consume Heartbeat data for meaningful alerting? **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-monitoring, elastic-stack-alerting\ **Created:** [February 18, 2020, 10:02pm UTC](https://discuss.elastic.co/t/how-do-i-consume-heartbeat-data-for-meaningful-alerting/219871 "2020-02-18T22:02:42Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![geoffmore](https://avatars.discourse-cdn.com/v4/letter/g/838e76/32.png) [@geoffmore](https://discuss.elastic.co/u/geoffmore)\ **Post date:** [February 18, 2020, 10:02pm UTC](https://discuss.elastic.co/t/how-do-i-consume-heartbeat-data-for-meaningful-alerting/219871/1 "2020-02-18T22:02:43Z")
I have created a Watcher that consumes the data from a named, icmp, Heartbeat monitor [1] running as a Kubernetes DaemonSet [2]. I have not yet gotten approval to post the configuration of the Watcher, so I will provide a high-level description of how I get the data. The watcher includes the following fields which are documented here [3] and here [4]: - agent.hostname - [rtt.us](http://rtt.us) - url.domain - monitor.type - monitor.name To get to my desired data: - Documents are filtered using a the "icmp" monitor type and a provided monitor name - Documents are separated into unique buckets using agent.hostname (source) and url.domain (dest) [5] - Once the documents are separated by into unique buckets with a max bucket count of 10k (this may be the maximum) - A max aggregation [6] is performed on each of those composite buckets Heartbeat is configured to create a ping every 5 seconds; both the Input [6] and Trigger [7] are configured to look back 5m, so I expect 60 documents per pair over that duration. On my largest Kubernetes cluster, there are 23 nodes, so I expect 60 \* 23^2 (~63.5k) documents across 23^2 \* 2 (~1k) buckets (23^2 for composite, x2 because each composite gets a max agg). Even though my max buckets is configured at 10k buckets, I have noticed that I get an after\_key [8] which truncates all pairs after around 16 (when iterating a-a -\> w-w). I would like to know what I should be doing to return the entire output for consumption by my Watcher. I would also like to know how to address the cases where I scale from 2 to n pages of output based on the number of buckets I'm returning. I believe the Chain Input [9] solves the problem with a known number of pages, but I am not sure if this is the best and/or most scalable approach. As an aside, if/when I am able to share this Watcher code, where would be the best public repo to keep it? [1] [https://www.elastic.co/guide/en/beats/heartbeat/7.6/configuration-heartbeat-options.html](https://www.elastic.co/guide/en/beats/heartbeat/7.6/configuration-heartbeat-options.html) [2] [https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) [3] [https://www.elastic.co/guide/en/beats/heartbeat/7.6/exported-fields-icmp.html](https://www.elastic.co/guide/en/beats/heartbeat/7.6/exported-fields-icmp.html) [4] [https://www.elastic.co/guide/en/beats/heartbeat/7.6/exported-fields-ecs.html](https://www.elastic.co/guide/en/beats/heartbeat/7.6/exported-fields-ecs.html) [5] [https://www.elastic.co/guide/en/elasticsearch/reference/7.6/search-aggregations-bucket-composite-aggregation.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/search-aggregations-bucket-composite-aggregation.html) [6] [https://www.elastic.co/guide/en/elasticsearch/reference/7.6/input.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/input.html) [7] [https://www.elastic.co/guide/en/elasticsearch/reference/7.6/trigger-schedule.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/trigger-schedule.html) [8] [https://www.elastic.co/guide/en/elasticsearch/reference/7.6/search-aggregations-bucket-composite-aggregation.html#\_pagination](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/search-aggregations-bucket-composite-aggregation.html#_pagination) [9] [https://www.elastic.co/guide/en/elasticsearch/reference/7.6/input-chain.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/input-chain.html) (edit: fixed formatting issue) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 17, 2020, 10:02pm UTC](https://discuss.elastic.co/t/how-do-i-consume-heartbeat-data-for-meaningful-alerting/219871/2 "2020-03-17T22:02:48Z")
Closed on March 17, 2020, 10:02pm UTC This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.