# Elastic rollover performance thoughts

**URL:** <https://discuss.elastic.co/t/elastic-rollover-performance-thoughts/233535>\
**Category:** Kibana\
**Created:** [May 20, 2020, 12:36pm UTC](https://discuss.elastic.co/t/elastic-rollover-performance-thoughts/233535 "2020-05-20T12:36:16Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![liorg2](https://avatars.discourse-cdn.com/v4/letter/l/ed8c4c/32.png) [@liorg2](https://discuss.elastic.co/u/liorg2)\
**Post date:** [May 20, 2020, 12:36pm UTC](https://discuss.elastic.co/t/elastic-rollover-performance-thoughts/233535/1 "2020-05-20T12:36:16Z")

</div>

Hello,  
in the past I used to have an index per month, and named them:  
logs-YYYY-MM  
when query them I used to build a list of indices, and concat them, with a date range filter.  
for example. if the date range is last 1.5 months, the request will be:  
GET logs-2020-04,logs-2020-05/\_search..

when started to use ILM, I changed that to query an index pattern with a date range filter.  
GET logs-\*/\_search

it leads me to think, that in case of a monthly index (30GB) with 1 replica,  
even if the date filter would be: last 3 days, the query will scan (potentially) shard of a few years.

My question is

1.how can I prevent this scan:  
elastic search supports pre\_filter\_shard\_size parameter  
but it seems that the pre\_filter\_shard\_size will be active when shard number \> 128, probably for performance reasons

1. how meaningful is that for Lucene (scanning a shard that has 0 documents relevant to date filter)

2. other thoughts?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 20, 2020, 11:01pm UTC](https://discuss.elastic.co/t/elastic-rollover-performance-thoughts/233535/2 "2020-05-20T23:01:51Z")

</div>

There's been a few improvements to Elasticsearch to handle this type of range query, and you don't need to worry about it reading all the data in all the shards, as it'll skip shards that don't contain relevant data.

---

<div class="post-metadata">

**Author:** ![liorg2](https://avatars.discourse-cdn.com/v4/letter/l/ed8c4c/32.png) [@liorg2](https://discuss.elastic.co/u/liorg2)\
**Post date:** [May 21, 2020, 4:28am UTC](https://discuss.elastic.co/t/elastic-rollover-performance-thoughts/233535/3 "2020-05-21T04:28:19Z")

</div>

Thanks a lot! Anywhere I can read about it?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 18, 2020, 4:28am UTC](https://discuss.elastic.co/t/elastic-rollover-performance-thoughts/233535/4 "2020-06-18T04:28:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.