# Help to increase number of shards on active index running close to 2B records **URL:** \ **Category:** Elasticsearch\ **Created:** [July 2, 2020, 12:50am UTC](https://discuss.elastic.co/t/help-to-increase-number-of-shards-on-active-index-running-close-to-2b-records/239560 "2020-07-02T00:50:55Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![saiguru](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/saiguru/32/51529_2.png) [@saiguru](https://discuss.elastic.co/u/saiguru)\ **Post date:** [July 2, 2020, 12:50am UTC](https://discuss.elastic.co/t/help-to-increase-number-of-shards-on-active-index-running-close-to-2b-records/239560/1 "2020-07-02T00:50:55Z")
I have 3 master and 2 data node with each 1TB cluster. I build a production index and it is reaching limit of 2B, how to increase # of shards to 10 on running index. ip heap.percent ram.percent cpu load\_1m load\_5m load\_15m node.role master name 10.21.17.8 81 99 52 1.92 1.52 1.42 dil - elk-data-vm3 10.21.17.5 26 66 0 0.00 0.01 0.00 ilm - elk-master-vm2 10.21.17.4 69 99 18 1.26 1.00 0.98 dil - elk-data-vm4 10.21.17.6 25 48 0 0.00 0.00 0.00 ilm - elk-master-vm0 10.21.17.7 35 64 1 0.00 0.02 0.00 ilm \* elk-master-vm1 epoch timestamp cluster status node.total node.data shards pri relo init unassign pending\_tasks max\_task\_wait\_time active\_shards\_percent 1593650988 00:49:48 aiopselkcluster red 5 2 71 36 0 0 19 0 - 78.9% yellow open insight\_tapm\_prod\_v1 dEEH71UwQe6enExvoKkpCw 1 2 1912941427 0 156gb 78gb ---
**Author:** ![AClerk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aclerk/32/55297_2.png) [@AClerk](https://discuss.elastic.co/u/AClerk)\ **Post date:** [July 2, 2020, 2:53am UTC](https://discuss.elastic.co/t/help-to-increase-number-of-shards-on-active-index-running-close-to-2b-records/239560/2 "2020-07-02T02:53:33Z")
AFAIK You can create an index template with the desired settings. Then re-index. But it sounds a bit odd that you have such a large index. Maybe consider re-designing your indexing solution? Thanks ---
**Author:** ![saiguru](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/saiguru/32/51529_2.png) [@saiguru](https://discuss.elastic.co/u/saiguru)\ **Post date:** [July 2, 2020, 4:21am UTC](https://discuss.elastic.co/t/help-to-increase-number-of-shards-on-active-index-running-close-to-2b-records/239560/3 "2020-07-02T04:21:02Z")
thanks for quick response, we are did not use template while creating index. We are working on large machine learning logs and data sets, is there a recommended pattern for template and index? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 2, 2020, 5:05am UTC](https://discuss.elastic.co/t/help-to-increase-number-of-shards-on-active-index-running-close-to-2b-records/239560/4 "2020-07-02T05:05:32Z")
Which version of Elasticsearch are you using? Are you just inserting new data or updating and deleting as well? ---
**Author:** ![saiguru](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/saiguru/32/51529_2.png) [@saiguru](https://discuss.elastic.co/u/saiguru)\ **Post date:** [July 2, 2020, 2:11pm UTC](https://discuss.elastic.co/t/help-to-increase-number-of-shards-on-active-index-running-close-to-2b-records/239560/5 "2020-07-02T14:11:09Z")
7.6.2 inserting new records every 5 min as we process data with ML models. Quick help needed is -- to increase shards on existing index, and i tried -- make index and read only and increase shards. Would like to know if i am making any mistake on above steps. ---
**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\ **Post date:** [July 2, 2020, 2:24pm UTC](https://discuss.elastic.co/t/help-to-increase-number-of-shards-on-active-index-running-close-to-2b-records/239560/6 "2020-07-02T14:24:39Z")
> [@saiguru](#): > > Quick help needed is With [this](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-split-index.html) changing shard size you can. Just kidding. You will have to split your index into a new one with more shards. First of all you will have to change the `number_of_routing_shards` setting to something like 25. After that set the index to read only and perform the split. ```auto POST /index/_split/split-index { "settings": { "index.number_of_shards": 20 } } ``` Cheers, _yoda._ ---
**Author:** ![saiguru](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/saiguru/32/51529_2.png) [@saiguru](https://discuss.elastic.co/u/saiguru)\ **Post date:** [July 2, 2020, 2:41pm UTC](https://discuss.elastic.co/t/help-to-increase-number-of-shards-on-active-index-running-close-to-2b-records/239560/7 "2020-07-02T14:41:54Z")
to set index as read only -- is below command right? PUT //\_settings { "index": { "blocks.read\_only": true blocks.read\_only\_allow\_delete": true } } ---
**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\ **Post date:** [July 2, 2020, 2:44pm UTC](https://discuss.elastic.co/t/help-to-increase-number-of-shards-on-active-index-running-close-to-2b-records/239560/8 "2020-07-02T14:44:59Z")
Yes, looks right to me. But be aware that you can't index anything into the index if you set it into read only (as the name suggests). You will have a downtime of a day or more if you havent set up counter measures. Spliting the index will take its time, becuase your index is huge. ---
**Author:** ![saiguru](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/saiguru/32/51529_2.png) [@saiguru](https://discuss.elastic.co/u/saiguru)\ **Post date:** [July 2, 2020, 2:48pm UTC](https://discuss.elastic.co/t/help-to-increase-number-of-shards-on-active-index-running-close-to-2b-records/239560/9 "2020-07-02T14:48:29Z")
oops good point, i can't do this real-time. thanks for pointing out. What is recommended action - option1: Create new index with more shards and push data there option2: downtime to create splits any more options? ---
**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\ **Post date:** [July 2, 2020, 2:53pm UTC](https://discuss.elastic.co/t/help-to-increase-number-of-shards-on-active-index-running-close-to-2b-records/239560/10 "2020-07-02T14:53:54Z")
My idea would be: 1. Create a new index with more shards and the right mapping. 2. Index new data into this new index. 3. `_reindex`the old index into the new one. I am not 100% sure this works so I would wait for @Christian_Dahlqvist, @dadoonet or another Elasticsearch Team member to verify these steps. You don't want to riskt a data loss. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 30, 2020, 2:53pm UTC](https://discuss.elastic.co/t/help-to-increase-number-of-shards-on-active-index-running-close-to-2b-records/239560/11 "2020-07-30T14:53:54Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.