# Performance querying time-based indices in a date range

**URL:** <https://discuss.elastic.co/t/performance-querying-time-based-indices-in-a-date-range/238283>\
**Category:** Elasticsearch\
**Created:** [June 23, 2020, 2:42pm UTC](https://discuss.elastic.co/t/performance-querying-time-based-indices-in-a-date-range/238283 "2020-06-23T14:42:31Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![devplayer0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/devplayer0/32/70907_2.png) [@devplayer0](https://discuss.elastic.co/u/devplayer0)\
**Post date:** [June 23, 2020, 2:42pm UTC](https://discuss.elastic.co/t/performance-querying-time-based-indices-in-a-date-range/238283/1 "2020-06-23T14:42:31Z")

</div>

Bit new to Elasticsearch. When limiting the date range of a query (for example if I have a daily index of events and I'd like to search within only a non-aligned monthly cycle), is it worth optimising the index pattern?

My application exposes a query API which is internally translated into an Elasticsearch query. If I want to query only in dates 2020-05-17 to 2020-06-17, should my application optimise the query knowing the format of index names:

```json
GET /events-2020.05.17,events-2020.05.18,<...>/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "someMetric": {
              "gte": 0.9
            }
          }
        }
      ]
    }
  }
}

```

vs

```json
GET /events-*/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "@timestamp": {
              "gte": "2020-05-17",
              "lt": "2020-06-17"
            }
          }
        },
        {
          "range": {
            "someMetric": {
              "gte": 0.9
            }
          }
        }
      ]
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 24, 2020, 4:46am UTC](https://discuss.elastic.co/t/performance-querying-time-based-indices-in-a-date-range/238283/2 "2020-06-24T04:46:21Z")

</div>

Elasticsearch is quite efficient when it comes to filtering out shards that do not have any data within a time range, so I would recommend testing to see what the difference is for your specific scenario as it may vary depending on data and number of indices and shards.

---

<div class="post-metadata">

**Author:** ![itizir](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itizir/32/63552_2.png) [@itizir](https://discuss.elastic.co/u/itizir)\
**Post date:** [July 6, 2020, 10:16am UTC](https://discuss.elastic.co/t/performance-querying-time-based-indices-in-a-date-range/238283/3 "2020-07-06T10:16:38Z")

</div>

Oh btw, if you investigate the performance of time range filters, I'd be interested to hear whether you also see this...

> [@@timestamp field slow range query "DocValuesFieldExistsQuery"](https://discuss.elastic.co/t/timestamp-field-slow-range-query-docvaluesfieldexistsquery/234995):
>
> Hi! I was profiling my queries and I run into this: According to [this](https://discuss.elastic.co/t/time-range-query-performance-7-6/223194/16) thread there's isn't anything I can do to avoid this behavior and it seems that the newest ES version has older Lucene version (8.5) which doesn't have the [fixed issue](https://issues.apache.org/jira/browse/LUCENE-9287) (since it's very recent). The field is mapped as 'date', ES version: 7.2.1. Edit: The daily index in question has 6 shards and the total avg size is ~260GB and has ~600M documents. The timestamp in the event is in seconds. Thanks!

> [@Time range query performance \[7.6\]](https://discuss.elastic.co/t/time-range-query-performance-7-6/223194/15):
>
> I am afraid you cannot disable parts on the query cache. On the other hand, I have discussed this behaviour with other people and we agreed that we should not cache DocValuesFieldExistsQuery as the query is already very fast and the overhead makes little sense. This has been already fixed in the upstream project: [https://issues.apache.org/jira/browse/LUCENE-9287](https://issues.apache.org/jira/browse/LUCENE-9287)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 3, 2020, 10:16am UTC](https://discuss.elastic.co/t/performance-querying-time-based-indices-in-a-date-range/238283/4 "2020-08-03T10:16:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.