# Index reallocation takes hours **URL:** \ **Category:** Elasticsearch\ **Created:** [October 7, 2014, 3:42pm UTC](https://discuss.elastic.co/t/index-reallocation-takes-hours/20119 "2014-10-07T15:42:46Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Andrew\_Lakes](https://avatars.discourse-cdn.com/v4/letter/a/c2a13f/32.png) [@Andrew\_Lakes](https://discuss.elastic.co/u/Andrew_Lakes)\ **Post date:** [October 7, 2014, 3:42pm UTC](https://discuss.elastic.co/t/index-reallocation-takes-hours/20119/1 "2014-10-07T15:42:46Z")
Hey guys, we have setup an central ES-server with 2 instances (1 instance SSDs for indexing/last indices and an other one for long term storage) Everyday at 6 a.m a cronjob is started that moves indices older than 7 days to the HDD storage node. Our indices have an average size of 300 GB, but it takes about 6-7 hours till the reallocation of 1 index is done (if you copy it manually with "cp" it only takes about 30 minutes). Is there any settings which can i adjust that speed this process of reallocation up? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/aa78aefb-be9e-4dc6-bdca-948d312f06af%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/aa78aefb-be9e-4dc6-bdca-948d312f06af%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [October 7, 2014, 5:35pm UTC](https://discuss.elastic.co/t/index-reallocation-takes-hours/20119/2 "2014-10-07T17:35:29Z")
Currently shard allocation is throttled by default with a target low value. If you are using an older version of Elasticsearch, then a Lucene bug will make it even slower. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. How many shards do your indices have? The default number of concurrent allocations is only 2. The ideal values depends on your hardware, especially your disk io. Cheers, Ivan On Oct 7, 2014 8:42 AM, "Andrew Lakes" [alakes501@gmail.com](mailto:alakes501@gmail.com) wrote: > Hey guys, > > we have setup an central ES-server with 2 instances (1 instance SSDs for > indexing/last indices and an other one for long term storage) > > Everyday at 6 a.m a cronjob is started that moves indices older than 7 > days to the HDD storage node. > > Our indices have an average size of 300 GB, but it takes about 6-7 hours > till the reallocation of 1 index is done (if you copy it manually with "cp" > it only takes about 30 minutes). > > Is there any settings which can i adjust that speed this process of > reallocation up? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/aa78aefb-be9e-4dc6-bdca-948d312f06af%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/aa78aefb-be9e-4dc6-bdca-948d312f06af%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/aa78aefb-be9e-4dc6-bdca-948d312f06af%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/aa78aefb-be9e-4dc6-bdca-948d312f06af%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAdR\_Fj\_kDu7vtg6pc5\_FkQ7A1UF6jR6tT56tVbPcxbsw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAdR_Fj_kDu7vtg6pc5_FkQ7A1UF6jR6tT56tVbPcxbsw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Andrew\_Lakes](https://avatars.discourse-cdn.com/v4/letter/a/c2a13f/32.png) [@Andrew\_Lakes](https://discuss.elastic.co/u/Andrew_Lakes)\ **Post date:** [October 8, 2014, 9:04am UTC](https://discuss.elastic.co/t/index-reallocation-takes-hours/20119/3 "2014-10-08T09:04:26Z")
Hi, we are using version 1.3.2 of ES , each index have 4 shards. I will try adjusting the store-throttling to an acceptable number without crushing our disk IOs Am Dienstag, 7. Oktober 2014 19:35:35 UTC+2 schrieb Ivan Brusic: > Currently shard allocation is throttled by default with a target low > value. If you are using an older version of Elasticsearch, then a Lucene > bug will make it even slower. > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-store.html#store-throttling) > > How many shards do your indices have? The default number of concurrent > allocations is only 2. The ideal values depends on your hardware, > especially your disk io. > > Cheers, > Ivan > On Oct 7, 2014 8:42 AM, "Andrew Lakes" \<[alak...@gmail.com](mailto:alak...@gmail.com) \\> > wrote: > > > Hey guys, > > > > we have setup an central ES-server with 2 instances (1 instance SSDs for > > indexing/last indices and an other one for long term storage) > > > > Everyday at 6 a.m a cronjob is started that moves indices older than 7 > > days to the HDD storage node. > > > > Our indices have an average size of 300 GB, but it takes about 6-7 hours > > till the reallocation of 1 index is done (if you copy it manually with "cp" > > it only takes about 30 minutes). > > > > Is there any settings which can i adjust that speed this process of > > reallocation up? > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/aa78aefb-be9e-4dc6-bdca-948d312f06af%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/aa78aefb-be9e-4dc6-bdca-948d312f06af%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/aa78aefb-be9e-4dc6-bdca-948d312f06af%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/aa78aefb-be9e-4dc6-bdca-948d312f06af%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/09f3f2db-0ae7-4f0a-8891-e4ab267d1c78%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/09f3f2db-0ae7-4f0a-8891-e4ab267d1c78%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![german23](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/german23/32/11052_2.png) [@german23](https://discuss.elastic.co/u/german23)\ **Post date:** [October 29, 2014, 12:46pm UTC](https://discuss.elastic.co/t/index-reallocation-takes-hours/20119/4 "2014-10-29T12:46:52Z")
Hi, i want to use this thread, to clarify some of the elasticsearch settings, because im struggeling a bit in understanding all of them. 1.indices.store.throttle.max\_bytes\_per\_sec : This is the maximum amount of mb/gb which elasticsearch uses per second to to the merging process - right? (Is there an difference between "indices.store.throttle.max\_bytes\_per\_sec" and "index.store.throttle.max\_bytes\_per\_sec"? ) 2.indices.recovery.max\_bytes\_per\_sec: This is the maximum amout of mb/gb which elasticsearch uses to initialize shards/indices after an restart of the node - right? 3.indices.recovery.concurrent\_streams: The amount of parallel shard initializations after an restart - right? 4.cluster.routing.allocation.node\_concurrent\_recoveries: The cluster wide amount of parallel shard initializations after an restart - right? 5.cluster.routing.allocation.node\_initial\_primaries\_recoveries: The cluster wide amount of parallel primary shard initializations after an restart - right? 6.cluster.routing.allocation.cluster\_concurrent\_rebalance:2: The amount of shards that are parallel allowed to move from one node to an other -right? But of all the settings found in the elasticsearch there is no such settings like "cluster.routing.allocation.max\_bytes\_per\_sec" which limits the maximum amount of mb/gb per sec that can be used to allocate a shard from one node to another - will elasticsearch by default use a value or just use the maximum possible speed? Thanks for response Am Mittwoch, 8. Oktober 2014 11:04:26 UTC+2 schrieb Andrew Lakes: > Hi, > > we are using version 1.3.2 of ES , each index have 4 shards. > > I will try adjusting the store-throttling to an acceptable number without > crushing our disk IOs > > Am Dienstag, 7. Oktober 2014 19:35:35 UTC+2 schrieb Ivan Brusic: > > > Currently shard allocation is throttled by default with a target low > > value. If you are using an older version of Elasticsearch, then a Lucene > > bug will make it even slower. > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-store.html#store-throttling) > > > > How many shards do your indices have? The default number of concurrent > > allocations is only 2. The ideal values depends on your hardware, > > especially your disk io. > > > > Cheers, > > Ivan > > On Oct 7, 2014 8:42 AM, "Andrew Lakes" [alak...@gmail.com](mailto:alak...@gmail.com) wrote: > > > > > Hey guys, > > > > > > we have setup an central ES-server with 2 instances (1 instance SSDs for > > > indexing/last indices and an other one for long term storage) > > > > > > Everyday at 6 a.m a cronjob is started that moves indices older than 7 > > > days to the HDD storage node. > > > > > > Our indices have an average size of 300 GB, but it takes about 6-7 hours > > > till the reallocation of 1 index is done (if you copy it manually with "cp" > > > it only takes about 30 minutes). > > > > > > Is there any settings which can i adjust that speed this process of > > > reallocation up? > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/aa78aefb-be9e-4dc6-bdca-948d312f06af%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/aa78aefb-be9e-4dc6-bdca-948d312f06af%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/aa78aefb-be9e-4dc6-bdca-948d312f06af%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/aa78aefb-be9e-4dc6-bdca-948d312f06af%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1224c3f7-8207-42f5-8d2d-95eb36280ef2%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/1224c3f7-8207-42f5-8d2d-95eb36280ef2%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:53am UTC](https://discuss.elastic.co/t/index-reallocation-takes-hours/20119/5 "2017-07-06T00:53:13Z")