# Aggregations / Curl / # of events by device in the past hour

**URL:** <https://discuss.elastic.co/t/aggregations-curl-of-events-by-device-in-the-past-hour/64636>\
**Category:** Elasticsearch\
**Created:** [November 1, 2016, 9:02pm UTC](https://discuss.elastic.co/t/aggregations-curl-of-events-by-device-in-the-past-hour/64636 "2016-11-01T21:02:52Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![l1carter](https://avatars.discourse-cdn.com/v4/letter/l/74df32/32.png) [@l1carter](https://discuss.elastic.co/u/l1carter)\
**Post date:** [November 1, 2016, 9:02pm UTC](https://discuss.elastic.co/t/aggregations-curl-of-events-by-device-in-the-past-hour/64636/1 "2016-11-01T21:02:52Z")

</div>

All, looking for a simple curl command that can run every hour in crontab. it's purpose is to look at the total number of events generated for the past hour and sort by: source device generating event, total number of events generated by that device.

we use ELK for network syslog monitoring. we have a few hundred devices pointed to a syslog-ng process and what I'm hoping to do is simply check every hour how many events each device generated. some will generate 0 events. some (firewalls) will generate a hundred or so... I just need to know how to curl that data out.

from there I'll have my script check the values to a known set of "acceptable ranges" and if they are out of that range it will send an e-mail alert. this piece I can figure out... it's the curl in elasticsearch that I could use some assistance on. any help / guidance is greatly appreciated.

thanks,

Lee

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 2, 2016, 4:57am UTC](https://discuss.elastic.co/t/aggregations-curl-of-events-by-device-in-the-past-hour/64636/2 "2016-11-02T04:57:30Z")

</div>

You'll want to do a [terms aggregation](https://www.elastic.co/guide/en/elasticsearch//reference/5.0/search-aggregations-bucket-terms-aggregation.html). Simple example:

```nohighlight
curl -XPOST hostname:9200/_search\?pretty -d '
{
  "size": 0,
  "aggs": {
    "any-name-for-this-agg": {
      "terms": {
        "field": "name-of-field-to-aggregate"
      }
    }
  }
}'

```

If you only want to check that last hour you'll need to add a restriction on the `@timestamp` field.

---

<div class="post-metadata">

**Author:** ![l1carter](https://avatars.discourse-cdn.com/v4/letter/l/74df32/32.png) [@l1carter](https://discuss.elastic.co/u/l1carter)\
**Post date:** [November 2, 2016, 3:17pm UTC](https://discuss.elastic.co/t/aggregations-curl-of-events-by-device-in-the-past-hour/64636/3 "2016-11-02T15:17:58Z")

</div>

thanks so much. I've been working with your framework and so far this works great:

```
curl -XPOST localhost:9200/_search\?pretty -d '
{
  "query": { "term" : { "devtype.raw" : "ASA Firewall" } },
    "aggs" : {
        "date_interval" : {
            "date_histogram" : {
                "field" : "syslog_server_time",
                "interval" : "month"
            }, "aggs": {
                   "any-name-for-this-agg": {
                       "terms": { "field": "hostname.raw" }
                   }
                }
            }
        }
    }
}'

```

but I think with the "two level's of agg"... what I'm getting is all events per month for every month... would like to get just the last month result... not every month prior... any thoughts/suggestions?

thanks,

Lee

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 2, 2016, 3:24pm UTC](https://discuss.elastic.co/t/aggregations-curl-of-events-by-device-in-the-past-hour/64636/4 "2016-11-02T15:24:09Z")

</div>

A date histogram is not what you're looking for. Add the date restriction to the query so that not all events are subject to the aggregation in the first place.

---

<div class="post-metadata">

**Author:** ![l1carter](https://avatars.discourse-cdn.com/v4/letter/l/74df32/32.png) [@l1carter](https://discuss.elastic.co/u/l1carter)\
**Post date:** [November 2, 2016, 6:21pm UTC](https://discuss.elastic.co/t/aggregations-curl-of-events-by-device-in-the-past-hour/64636/5 "2016-11-02T18:21:23Z")

</div>

thanks so much for your help on this... think I got what I need... need to still tweak it a bit but what I now have is:

```
curl -XPOST localhost:9200/_search\?pretty -d '
{
  "size": 0,
  "aggs": {
    "last_X_min": {
       "filter" : { 
          "range" : { 
             "syslog_server_time" : { 
                "gte": "now-15m", "lte" : "now" **<--- 15m to 5m changes how far back to look...**
              }
           }
        },
        "aggs" : {
           "any-name-for-this-agg" : {
              "terms" : {
                 "field" : "hostname.raw", **<--- finding I have to use field.raw for this to work... not sure why...**
                 "size":355 **<--- this number has to be greater than the total # of devices or you may miss one...**
               }
            }
         }
     }
   }
}'

```

thanks again.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [November 2, 2016, 6:26pm UTC](https://discuss.elastic.co/t/aggregations-curl-of-events-by-device-in-the-past-hour/64636/6 "2016-11-02T18:26:31Z")

</div>

I'm not very good with the query DSL and I don't have time to look things up, but as I said I think you should use the _query_ section for weeding out old events. But if you have above works, great.

> finding I have to use field.raw for this to work... not sure why

That's because the hostname field is analyzed. The .raw subfield isn't.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:07pm UTC](https://discuss.elastic.co/t/aggregations-curl-of-events-by-device-in-the-past-hour/64636/7 "2017-07-05T22:07:13Z")

</div>