# Important factor on the required time for opening a closed shard? **URL:** \ **Category:** Elasticsearch\ **Created:** [July 20, 2019, 7:36am UTC](https://discuss.elastic.co/t/important-factor-on-the-required-time-for-opening-a-closed-shard/191469 "2019-07-20T07:36:04Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Ali\_Reza\_Haghighat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ali_reza_haghighat/32/48590_2.png) [@Ali\_Reza\_Haghighat](https://discuss.elastic.co/u/Ali_Reza_Haghighat)\ **Post date:** [July 20, 2019, 7:36am UTC](https://discuss.elastic.co/t/important-factor-on-the-required-time-for-opening-a-closed-shard/191469/1 "2019-07-20T07:36:04Z")
I'd like to know the important factor on the time of opening a closed shard. I've run a few tests, but I got unreasonable results: No of Shards, Size of Each Shard, Opening Time 50, 10 MB, 5 s 50, 150 MB, 12 s 50, 2 GB, 8 s I expect that increasing size of shard affects the opening time, But the result of my result was different. I run the test multiple times and get a similar result. Could anyone help me in finding the important factor on the required time for opening a closed shard? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 20, 2019, 8:30am UTC](https://discuss.elastic.co/t/important-factor-on-the-required-time-for-opening-a-closed-shard/191469/2 "2019-07-20T08:30:49Z")
I would expect the opening time to depend on the mappings as well as the number of segments. Certain types of mappings require data structures to be constructed in memory, which can take time. Are there any major difference in mappings and/or segment count between your indices? ---
**Author:** ![Ali\_Reza\_Haghighat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ali_reza_haghighat/32/48590_2.png) [@Ali\_Reza\_Haghighat](https://discuss.elastic.co/u/Ali_Reza_Haghighat)\ **Post date:** [July 20, 2019, 1:28pm UTC](https://discuss.elastic.co/t/important-factor-on-the-required-time-for-opening-a-closed-shard/191469/3 "2019-07-20T13:28:19Z")
Thanks Christian, I have filled the shards with the same data. They have the same mappings. I think that the number of segments depends on the size of shard, i.e. larger shards have larger number of segments. Is it true? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 20, 2019, 4:59pm UTC](https://discuss.elastic.co/t/important-factor-on-the-required-time-for-opening-a-closed-shard/191469/4 "2019-07-20T16:59:11Z")
Opening a set of shards will result in a number of cluster state updates, so that may also play a part. ---
**Author:** ![Ali\_Reza\_Haghighat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ali_reza_haghighat/32/48590_2.png) [@Ali\_Reza\_Haghighat](https://discuss.elastic.co/u/Ali_Reza_Haghighat)\ **Post date:** [July 21, 2019, 8:56am UTC](https://discuss.elastic.co/t/important-factor-on-the-required-time-for-opening-a-closed-shard/191469/5 "2019-07-21T08:56:39Z")
Thanks Christian! But I've run the tests in almost similar conditions (at the same time, with the same number of nodes and indices). ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 21, 2019, 9:10am UTC](https://discuss.elastic.co/t/important-factor-on-the-required-time-for-opening-a-closed-shard/191469/6 "2019-07-21T09:10:36Z")
I can not explain the differences, but the tests are not very realistic anyway. All those indices are so small they really should have only a single primary shard. What is it you are looking to achieve? What is the purpose of this test? ---
**Author:** ![Ali\_Reza\_Haghighat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ali_reza_haghighat/32/48590_2.png) [@Ali\_Reza\_Haghighat](https://discuss.elastic.co/u/Ali_Reza_Haghighat)\ **Post date:** [July 21, 2019, 11:23am UTC](https://discuss.elastic.co/t/important-factor-on-the-required-time-for-opening-a-closed-shard/191469/7 "2019-07-21T11:23:47Z")
Let me describe our final goal. We use elastic to store time-series data. We need to store data for a specific time (e.g., 12 months). We have enough storage to store our data, however, we have not enough RAM to follow the recommendation of this [post](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster) (i.e., having 40 GB shards and 1 GB JVM heap per 20 shards). Therefore, we decide to use the frozen index for some of the old data (e.g., 5 months of data) or manually close and open them when there is a query on them. Now, we want to know the optimal size of the shard to open our closed indices as fast as possible. P.S. I had run similar tests with larger shard sizes (e.g., 20 GB), however, I did not find any relation between the size of a shard and required time to open it. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 21, 2019, 11:44am UTC](https://discuss.elastic.co/t/important-factor-on-the-required-time-for-opening-a-closed-shard/191469/8 "2019-07-21T11:44:10Z")
> [@Ali\_Reza\_Haghighat](#): > > We have enough storage to store our data, however, we have not enough RAM to follow the recommendation of this [post](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster) (i.e., having 40 GB shards and 1 GB JVM heap per 20 shards). I am not sure I follow the reasoning. If you want to optimize heap usage you should make sure you have as large shards as possible as that generally uses less heap per document. I would further recommend setting aside dedicated nodes for long term storage and use frozen indices rather than closing them. Mke sure you forcemerge down to a single segment before you close or freeze the indices, as that will make opening them faster. Have a look at [this webinar](https://www.elastic.co/webinars/optimizing-storage-efficiency-in-elasticsearch) for additional details. This will allow you to open and search large shards reasonably quickly. ---
**Author:** ![Ali\_Reza\_Haghighat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ali_reza_haghighat/32/48590_2.png) [@Ali\_Reza\_Haghighat](https://discuss.elastic.co/u/Ali_Reza_Haghighat)\ **Post date:** [July 22, 2019, 10:19am UTC](https://discuss.elastic.co/t/important-factor-on-the-required-time-for-opening-a-closed-shard/191469/9 "2019-07-22T10:19:34Z")
Thank you, I will take a look at that webinar. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 19, 2019, 10:29am UTC](https://discuss.elastic.co/t/important-factor-on-the-required-time-for-opening-a-closed-shard/191469/10 "2019-08-19T10:29:31Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.