# Will many narrow index patterns improve search performance? **URL:** \ **Category:** Elasticsearch\ **Created:** [May 11, 2017, 9:40am UTC](https://discuss.elastic.co/t/will-many-narrow-index-patterns-improve-search-performance/85359 "2017-05-11T09:40:59Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Vitaly\_il](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vitaly_il/32/130964_2.png) [@Vitaly\_il](https://discuss.elastic.co/u/Vitaly_il)\ **Post date:** [May 11, 2017, 9:40am UTC](https://discuss.elastic.co/t/will-many-narrow-index-patterns-improve-search-performance/85359/1 "2017-05-11T09:40:59Z")
We're planning to migrate from single daily index to many indices in order to implement granular access policies. I.e. instead of logstash-DD.MM.YYYY we're going to have about 40 daily indices like logstash-applicaitonXX-DD.MM.YYYY. Should we add new index templates per application in addition to the default "logstash-\*" in order to avoid scanning unnecessary indices? TIA, Vitaly ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 11, 2017, 10:27am UTC](https://discuss.elastic.co/t/will-many-narrow-index-patterns-improve-search-performance/85359/2 "2017-05-11T10:27:56Z")
How much data are you ingesting per day? What is your retention period? ---
**Author:** ![Vitaly\_il](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vitaly_il/32/130964_2.png) [@Vitaly\_il](https://discuss.elastic.co/u/Vitaly_il)\ **Post date:** [May 11, 2017, 10:34am UTC](https://discuss.elastic.co/t/will-many-narrow-index-patterns-improve-search-performance/85359/3 "2017-05-11T10:34:24Z")
We're running Elastic on a standalone node with 16GB RAM, with 30 days retention, we have about 1300 indices, about 800GB data. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 11, 2017, 1:57pm UTC](https://discuss.elastic.co/t/will-many-narrow-index-patterns-improve-search-performance/85359/4 "2017-05-11T13:57:57Z")
That sounds like a very large number of indices and shards given the cluster size and data volume. The default Logstash index template applies to indices starting with `logstash-`, so would apply to all indices following the naming convention you outlined. ---
**Author:** ![Vitaly\_il](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vitaly_il/32/130964_2.png) [@Vitaly\_il](https://discuss.elastic.co/u/Vitaly_il)\ **Post date:** [May 11, 2017, 2:35pm UTC](https://discuss.elastic.co/t/will-many-narrow-index-patterns-improve-search-performance/85359/5 "2017-05-11T14:35:26Z")
Thank you! So if we'll create new index templates ("logstash-nginx-\*" and so on) it should decrease number of indices involved in search? Or there is no choice and we should decrease number of indices? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 11, 2017, 4:12pm UTC](https://discuss.elastic.co/t/will-many-narrow-index-patterns-improve-search-performance/85359/6 "2017-05-11T16:12:25Z")
The index template does not affect which indices that are included in a search, only how the data is mapped. If the current Logstash mappings are fine and you do not need to optimise mappings per application, you can continue using the default one. In order to limit the indices being queried, you need to create more specific index patterns in Kibana and use these when you build dashboards. This also applies if you are querying in other ways. Having that many indices and shards is going to result in overhead, as each shard requires memory and file handles, so it is not a given that your query performance will improve, especially if you need to hit a lot of shards. ---
**Author:** ![Vitaly\_il](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vitaly_il/32/130964_2.png) [@Vitaly\_il](https://discuss.elastic.co/u/Vitaly_il)\ **Post date:** [May 11, 2017, 6:37pm UTC](https://discuss.elastic.co/t/will-many-narrow-index-patterns-improve-search-performance/85359/7 "2017-05-11T18:37:24Z")
Of course, I meant Kibana index patterns. Thank you again, we'll try to play with our configuration. Vitaly ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 8, 2017, 6:50pm UTC](https://discuss.elastic.co/t/will-many-narrow-index-patterns-improve-search-performance/85359/8 "2017-06-08T18:50:45Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.