# Why the number of closed index affects write performance? **URL:** \ **Category:** Elasticsearch\ **Created:** [August 23, 2021, 12:10pm UTC](https://discuss.elastic.co/t/why-the-number-of-closed-index-affects-write-performance/282219 "2021-08-23T12:10:23Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![wulixuan](https://avatars.discourse-cdn.com/v4/letter/w/f07891/32.png) [@wulixuan](https://discuss.elastic.co/u/wulixuan)\ **Post date:** [August 23, 2021, 12:10pm UTC](https://discuss.elastic.co/t/why-the-number-of-closed-index-affects-write-performance/282219/1 "2021-08-23T12:10:23Z")
Hello: I am now testing the write performance of ES(v7.13.4), the number of shards written is constant, I added many empty indexes(shard number is 3). Then I find that the write performance is degraded even if the index is closed. Indices metadata and routing table are all map structures, this does not affect. Please tell me what caused it. The test results are as follows: Index number: 5790 Total shard number: 7974 Open shard: 7974 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/f/1fbc9cfc048e889639b8983ee79175d692a906f6.png) Index number: 10287 Total shard number: 17829 Open shard: 7974 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/4/74b3649e1c29cffb7963991c361579e2815d3147.png) Index number: 22126 Total shard number: 38524 Open shard: 7974 ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/7/87d923b7a9a7e200efe9b1f81c9fa06b138ef629.png) ---
**Author:** ![wulixuan](https://avatars.discourse-cdn.com/v4/letter/w/f07891/32.png) [@wulixuan](https://discuss.elastic.co/u/wulixuan)\ **Post date:** [August 23, 2021, 12:14pm UTC](https://discuss.elastic.co/t/why-the-number-of-closed-index-affects-write-performance/282219/2 "2021-08-23T12:14:42Z")
Version 5.X does not have this problem ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 23, 2021, 10:09pm UTC](https://discuss.elastic.co/t/why-the-number-of-closed-index-affects-write-performance/282219/3 "2021-08-23T22:09:50Z")
Why are you creating empty shards and then closing them? How many nodes in the cluster? ---
**Author:** ![wulixuan](https://avatars.discourse-cdn.com/v4/letter/w/f07891/32.png) [@wulixuan](https://discuss.elastic.co/u/wulixuan)\ **Post date:** [August 24, 2021, 2:55am UTC](https://discuss.elastic.co/t/why-the-number-of-closed-index-affects-write-performance/282219/4 "2021-08-24T02:55:36Z")
This is a test, because there are many closed indexes when used online. The cluster has 27 nodes, 3 of which are master nodes. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 24, 2021, 3:34am UTC](https://discuss.elastic.co/t/why-the-number-of-closed-index-affects-write-performance/282219/5 "2021-08-24T03:34:19Z")
In recent versions Elasticsearch is keeping track of closed indices, so I assume a large number of closed indices would increase the size of the cluster state more than before. I assume this could potentially slow down cluster state updates, if you create new indices as part of your indexing or use dynamic mappings which result in mapping updates to the cluster state. Are you using dynamic mappings? What does indexing performance look like if you index data into a few existing indices in a static format that does not result in new mappings? Why do you have so many closed indices in your cluster in the first place? ---
**Author:** ![wulixuan](https://avatars.discourse-cdn.com/v4/letter/w/f07891/32.png) [@wulixuan](https://discuss.elastic.co/u/wulixuan)\ **Post date:** [August 24, 2021, 3:52am UTC](https://discuss.elastic.co/t/why-the-number-of-closed-index-affects-write-performance/282219/6 "2021-08-24T03:52:43Z")
I tested the write performance after the index was created, and dynamic mapping is false. We opened the index for 3 days and kept the data for 14 days. If we need to search the data 3 days ago, we only need to open these indexes ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 24, 2021, 3:56am UTC](https://discuss.elastic.co/t/why-the-number-of-closed-index-affects-write-performance/282219/7 "2021-08-24T03:56:42Z")
Memory consumption related to shards have shrunk in recent version compared to e.g. Elasticsearch 5.x, so if resource usage is the reason you are closing indices this may no longer be required. Creating lots of.small indices and shards are however quite inefficient. Why are you creating so many small indices s as no shards? ---
**Author:** ![wulixuan](https://avatars.discourse-cdn.com/v4/letter/w/f07891/32.png) [@wulixuan](https://discuss.elastic.co/u/wulixuan)\ **Post date:** [August 24, 2021, 4:13am UTC](https://discuss.elastic.co/t/why-the-number-of-closed-index-affects-write-performance/282219/8 "2021-08-24T04:13:19Z")
Can this new feature be made into a parameter so that we can choose ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 24, 2021, 4:13am UTC](https://discuss.elastic.co/t/why-the-number-of-closed-index-affects-write-performance/282219/9 "2021-08-24T04:13:48Z")
No, it is not configurable. ---
**Author:** ![wulixuan](https://avatars.discourse-cdn.com/v4/letter/w/f07891/32.png) [@wulixuan](https://discuss.elastic.co/u/wulixuan)\ **Post date:** [August 24, 2021, 4:16am UTC](https://discuss.elastic.co/t/why-the-number-of-closed-index-affects-write-performance/282219/10 "2021-08-24T04:16:27Z")
Thank you, then we can only change it ourselves. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 24, 2021, 4:21am UTC](https://discuss.elastic.co/t/why-the-number-of-closed-index-affects-write-performance/282219/11 "2021-08-24T04:21:48Z")
It sounds like you are using Elasticsearch in an unusual way that is not necessarily following best practices. If you could describe your use case the community might be able to find a better way for you to shard and manage your data. If I recall correctly the change was introduces around the time when frozen indices were introduced. Before this closed indices were not at all monitored by the cluster and a node failure would not cause closed indices to get replicated to new nodes. For some use cases that relied on closed indices for handling very old data a couple of nodes failung could lead to the loss of all shard copies for specific shards, resulting in permanent data loss and red indices.. This is what I believe the change aimed at correcting. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 21, 2021, 4:21am UTC](https://discuss.elastic.co/t/why-the-number-of-closed-index-affects-write-performance/282219/12 "2021-09-21T04:21:58Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.