Help with date_histogram aggs over field by field

Elastic Stack Elasticsearch
1

Hi,

I'm trying to create an aggregate of an index that contains queueing metrics. Each document will contain, the queue name, enqueue rate, dequeue rate and some other values.

I created an aggregate query for the enqueue rate
POST /logstash-activemq-2017.07.31/_search?size=0
{
"aggs" : {
"aggs_60m" : {
"date_histogram": {
"field": "@timestamp",
"interval": "60m"
},
"aggs" : {
"enqueue_rate" : {
"stats" : { "field": "enqueue" }
}
}
}
}
}

This aggregates all queues together and not per queue. What I would like to is to create an aggregate that is.
60 minute stats of Enqueue Rate by Queue.

I'm at a loss on how to nest the, By Queue, into my query.
Thanks for the help,
Tim

2

Nest your date_histogram aggregation inside a terms aggregation on the queue name field. For example, if that field is called name, this request should give you what you want:

POST /logstash-activemq-2017.07.31/_search?size=0
{
  "aggs": {
    "queue": {
      "terms": {
        "field": "name",
        "size": 100
      }, 
      "aggs": {
        "aggs_60m": {
          "date_histogram": {
            "field": "@timestamp",
            "interval": "60m"
          },
          "aggs": {
            "enqueue_rate": {
              "stats": {
                "field": "enqueue"
              }
            }
          }
        }
      }
    }
  }
}

This will guve you a date histogram per queue.

Some things to be aware of here:

  • the name field needs to be mapped as type keyword. If you haven't explicitly mapped your fields, you should have a .keyword multi-field that you could use like this: "field": "name.keyword"
  • because of the "size": 100 you will only get the 100 most common queues (the queues for which there are the most documents). You can change the size parameter if you want to see more or less queues.
  • the queues will be ranked by popularity (how common they are - based on the doc_count). If you want, you can order the results differently.
3

Thanks that helps.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.