Speed-up Aggregation Data Histogram Aggregation query for static indices

Elastic Stack Elasticsearch
1

Hi,

I am using Elasticsearch 2.3 in the following scenario:

  • Log events are indexed in separate indices hourly. For example, all event related to 16th July 2017 14-15pm indexed in "events_2017.07.16.14".
  • Ingestion rate is about 5000 events per second. So each index contains about 18m documents.
  • Each index contains 4 shards and replication 1 (one extra copy)

My situation is I am using Date Histogram Aggregation query for the period of last 30 days frequently. Unfortunately, each query takes about 20 seconds, and it is not acceptable. Generally, for my problem all the indices are read-only, so the result for all the indices except the latest index would be constant. I was wondering how I can change Elasticsearch behaviour to ignore recalculating the Aggregation Query for the static indices? I've tried to enable request caching by changing index settings and change query end date from "now" to "now/d". Unfortunately, it didn't help me and the still query time is not acceptable.

Cheers,
Ali

2

You should upgrade. 5.X has caching for aggregations, but 2.X will recalculate it every time.

3

So there isn't any way I can have an aggregation caching with 2.3?

4

Not natively.

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.