Performance querying time-based indices in a date range

Elastic Stack Elasticsearch
1

Bit new to Elasticsearch. When limiting the date range of a query (for example if I have a daily index of events and I'd like to search within only a non-aligned monthly cycle), is it worth optimising the index pattern?

My application exposes a query API which is internally translated into an Elasticsearch query. If I want to query only in dates 2020-05-17 to 2020-06-17, should my application optimise the query knowing the format of index names:

GET /events-2020.05.17,events-2020.05.18,<...>/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "someMetric": {
              "gte": 0.9
            }
          }
        }
      ]
    }
  }
}

vs

GET /events-*/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "@timestamp": {
              "gte": "2020-05-17",
              "lt": "2020-06-17"
            }
          }
        },
        {
          "range": {
            "someMetric": {
              "gte": 0.9
            }
          }
        }
      ]
    }
  }
}
2

Elasticsearch is quite efficient when it comes to filtering out shards that do not have any data within a time range, so I would recommend testing to see what the difference is for your specific scenario as it may vary depending on data and number of indices and shards.

3

Oh btw, if you investigate the performance of time range filters, I'd be interested to hear whether you also see this...

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.