# Route search request by using the date in index name **URL:** \ **Category:** Elasticsearch\ **Created:** [April 7, 2021, 10:47pm UTC](https://discuss.elastic.co/t/route-search-request-by-using-the-date-in-index-name/269533 "2021-04-07T22:47:07Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![india](https://avatars.discourse-cdn.com/v4/letter/i/b487fb/32.png) [@india](https://discuss.elastic.co/u/india)\ **Post date:** [April 7, 2021, 10:47pm UTC](https://discuss.elastic.co/t/route-search-request-by-using-the-date-in-index-name/269533/1 "2021-04-07T22:47:07Z")
Hi, I am testing ILM, this is my configuration: ILM Policy: ```auto PUT _ilm/policy/test_policy { "policy": { "phases": { "hot": { "actions": { "rollover": { "max_docs": 1000 } } } }} } ``` Index template: ```auto PUT _index_template/test_template { "index_patterns": [ "test-*" ], "template": { "settings": { "index.lifecycle.name": "test_policy", "index.lifecycle.rollover_alias": "test_alias" }, "mappings": { "properties": { "testfield": { "type": "keyword" } } } } } ``` Bootstrap index: ```auto PUT / { "aliases": { "test_alias": { "is_write_index": true } } } ``` With this configuration, on every rollover new index is created, and since I am using date math in index name, every index contains rollover time in its name. I would like to pass date or date range along with my search request, so that search request would hit only index or indices that contain data matching the date or date range I have provided. That would be done by doing the math based on rollover time in index name and date I have provided with my search request. Is there such option in Elasticsearch? Thanks. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [April 7, 2021, 10:57pm UTC](https://discuss.elastic.co/t/route-search-request-by-using-the-date-in-index-name/269533/2 "2021-04-07T22:57:45Z")
That won't work with ILM, because every time that a policy rolls over it increments the counter on the end of the index name, it doesn't change the timestamp. ---
**Author:** ![india](https://avatars.discourse-cdn.com/v4/letter/i/b487fb/32.png) [@india](https://discuss.elastic.co/u/india)\ **Post date:** [April 8, 2021, 2:08pm UTC](https://discuss.elastic.co/t/route-search-request-by-using-the-date-in-index-name/269533/3 "2021-04-08T14:08:20Z")
It seems like it increments the counter, but also sets the rollover time in index name since I have used date math in bootstrap index. Here is a list of indices after few rollovers: ```auto GET /*/_alias/test_alias { "test-2021-04-08.09.27.23-000002" : { "aliases" : { "test_alias" : { "is_write_index" : false } } }, "test-2021-04-08.09.47.22-000003" : { "aliases" : { "test_alias" : { "is_write_index" : false } } }, "test-2021-04-08.10.07.22-000004" : { "aliases" : { "test_alias" : { "is_write_index" : true } } }, "test-2021-04-08.09.15.34-000001" : { "aliases" : { "test_alias" : { "is_write_index" : false } } } } ``` So, test-2021-04-08.09.47.22-000003 index contains data (logs in my case) with @timestamp in time range 09h 47m - 10h 07m. I am wondering if there is some kind of query where I could say 'I need logs with timestamp between 09h 50min and 10h 00m', and Elasticsearch would use rollover time in indices names to route that request to test-2021-04-08.09.47.22-000003 index and run search request only against that index. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 8, 2021, 2:33pm UTC](https://discuss.elastic.co/t/route-search-request-by-using-the-date-in-index-name/269533/4 "2021-04-08T14:33:10Z")
> [@india](#): > > I am wondering if there is some kind of query where I could say 'I need logs with timestamp between 09h 50min and 10h 00m', and Elasticsearch would use rollover time in indices names to route that request to test-2021-04-08.09.47.22-000003 index and run search request only against that index. Elasticsearch does this, only not using the index name which might not be right anyway: it just looks at the range of timestamps in all the relevant shards and skips any shards that don't match the range in the query. It's a very cheap check to make, and saves any of this hassle: just search `test-*` and let Elasticsearch pick the right shards. ---
**Author:** ![india](https://avatars.discourse-cdn.com/v4/letter/i/b487fb/32.png) [@india](https://discuss.elastic.co/u/india)\ **Post date:** [April 8, 2021, 9:03pm UTC](https://discuss.elastic.co/t/route-search-request-by-using-the-date-in-index-name/269533/5 "2021-04-08T21:03:03Z")
Hi, is there some additional setting/configuration I should set to enable that feature? I have added @timestamp field in my mapping, indexed some docs, and after few rollovers I tried to use range queries on @timestamp against test-\* but I do not see any shards being skipped. ---
**Author:** ![india](https://avatars.discourse-cdn.com/v4/letter/i/b487fb/32.png) [@india](https://discuss.elastic.co/u/india)\ **Post date:** [April 9, 2021, 10:11am UTC](https://discuss.elastic.co/t/route-search-request-by-using-the-date-in-index-name/269533/6 "2021-04-09T10:11:37Z")
Here is some additional info. When I add sort on @timestamp in query, I can see that some shards are being skipped: ```auto GET test-*/_search {"profile": "true", "query": {"range": { "@timestamp": { "gte": "2021-04-08T10:00:11.473Z", "lte": "2021-04-08T14:00:11.473Z" } }} , "sort": [ { "@timestamp": { "order": "desc" } } ] } ``` Result: ```auto "took" : 1, "timed_out" : false, "_shards" : { "total" : 4, "successful" : 4, "skipped" : 2, "failed" : 0 } ``` Without sorting on @timestamp field, there are no skipped shards, but query profiling is showing this for shards that should be skipped: ```auto "type" : "MatchNoDocsQuery", "description" : """MatchNoDocsQuery("User requested "match_none" query.")""", ``` Is it necessary to use sort on timestamp field if you want ES to skip shards that do not fit in time range? What does MatchNoDocsQuery means? Thanks. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 9, 2021, 11:00am UTC](https://discuss.elastic.co/t/route-search-request-by-using-the-date-in-index-name/269533/7 "2021-04-09T11:00:20Z")
It means that 2 shards won't have any of the data you are looking for so it's safe not to run the query against them. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 9, 2021, 12:19pm UTC](https://discuss.elastic.co/t/route-search-request-by-using-the-date-in-index-name/269533/8 "2021-04-09T12:19:59Z")
> [@india](#): > > What does MatchNoDocsQuery means? What David said, but in other words, it means these shards are effectively being skipped too. A `MatchNoDocsQuery` matches no documents in the shard, and as you might imagine it doesn't take much time or effort to execute that. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 10, 2021, 12:54pm UTC](https://discuss.elastic.co/t/route-search-request-by-using-the-date-in-index-name/269533/9 "2021-04-10T12:54:06Z")
I updated my post. Instead of > it's safe to run the query against them I actually meant: > it's safe **not** to run the query against them 🙂 ---
**Author:** ![liorg2](https://avatars.discourse-cdn.com/v4/letter/l/ed8c4c/32.png) [@liorg2](https://discuss.elastic.co/u/liorg2)\ **Post date:** [April 22, 2021, 1:38pm UTC](https://discuss.elastic.co/t/route-search-request-by-using-the-date-in-index-name/269533/10 "2021-04-22T13:38:55Z")
just found this post, as I'm trying to figure out something similar. I was wondering why the following query, returned: **"skipped" : 0** ``` POST /name-*/_search? { "query": { "bool": { "filter": [ { "range": { "datetime": { "gt": "now-2d" } } } ] } } } "took" : 29, "timed_out" : false, "_shards" : { "total" : 8, "successful" : 8, "skipped" : 0, "failed" : 0 }, ``` but when running the validateapi with the same query I got for 7 shards the same explanation: ``` "index" : "name-000001", "valid" : true, "explanation" : """MatchNoDocsQuery("User requested "match_none" query.")""" ``` and 1 shard actually had data: ``` "explanation" : "#DateRangeIncludingNowQuery(datetime:[1618925727715 TO 9223372036854775807])" ``` so what's the difference between skipped:7 to skipped:0 AND 7 shards returning MatchNoDocsQuery ? thanks! ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 22, 2021, 1:43pm UTC](https://discuss.elastic.co/t/route-search-request-by-using-the-date-in-index-name/269533/11 "2021-04-22T13:43:08Z")
> [@liorg2](#): > > so what's the difference between skipped:7 to skipped:0 AND 7 shards returning MatchNoDocsQuery ? Really it's just the phase at which the skipping takes place. We always try and rewrite the query to a `MatchNoDocsQuery` if possible, but sometimes this happens in a preflight check (resulting in `skipped` shards) and sometimes it happens at query time, depending on which is predicted to be more efficient. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 20, 2021, 1:44pm UTC](https://discuss.elastic.co/t/route-search-request-by-using-the-date-in-index-name/269533/12 "2021-05-20T13:44:04Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.