Multiple aggregation in one request vs one aggregation per request performance

[Mukolajko25 (Mykola Borysiuk)]

Hi.

Im using aggregations to compute data analysis. I use query to filter results and then a lot of aggregations to get data i want.

For example

"query": {
 "filtered": {
  "filter": {
    "bool": {
      "must": [
        {
          "range": {
            "ticketing.summary.counts.orders": {
              "from": 3
            }
          }
        },
        {
          "type": {
            "value": "profile"
          }
        }
      ]
    }
  }
 }
},
aggs: {20-30 terms and nested aggregations here}

I have a question about performance. Is it correct approach to run all aggregations in one request. Or maybe i need to run 1-2 aggregation per request. So from my code snipped i will have to run 15 requests instead of 1.

Elastic search version is 2.4
RAM : 5 GB for heap.
Documents : 500K-700K

Thanks for help.

[spinscale (Alexander Reelsen)]

Hey,

by splitting up, you could parallelize those aggregations across several requests. This however only makes sense, if there are no other requests doing search while you are running your aggregations.

As usual, the easiest way would be testing in your concrete use-case.

Also, you can use the shard request cache to cache responses, which might speed things up, if you have the same aggregations over and over again.

--Alex

[Mukolajko25 (Mykola Borysiuk)]

Hi. Thanks for help.

I agree that testing is the easiest way to check. However I hoped that elastic search already have answers about this kind of issues. My query timing are OK, but i wonder maybe elastic search doing better with a lot of small aggregations, or it's doesn't care as they always will have same time to execute. Like time to execute 100 es request to make 1 aggregations at a time === time to execute 1 es request with 100 aggregations in it? What is better for es itself? Thank you.