# Shard relocation progress **URL:** \ **Category:** Elasticsearch\ **Created:** [December 18, 2013, 9:28pm UTC](https://discuss.elastic.co/t/shard-relocation-progress/14940 "2013-12-18T21:28:33Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Mohit\_Anchlia](https://avatars.discourse-cdn.com/v4/letter/m/848f3c/32.png) [@Mohit\_Anchlia](https://discuss.elastic.co/u/Mohit_Anchlia)\ **Post date:** [December 18, 2013, 9:28pm UTC](https://discuss.elastic.co/t/shard-relocation-progress/14940/1 "2013-12-18T21:28:33Z")
What's the best way to monitor shard relocation that occurrs when one add new nodes? Is there a way to control the relocation and do it manually with few shards at a time? What are the best practices for cluster that is contantly received high volume traffic? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAOT3TWovrNe4cJvqAuvPV26zErezm0eqo9rF%2B%3DFpFk\_Pzh-q0Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAOT3TWovrNe4cJvqAuvPV26zErezm0eqo9rF%2B%3DFpFk_Pzh-q0Q%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Jason\_Wee](https://avatars.discourse-cdn.com/v4/letter/j/7ea924/32.png) [@Jason\_Wee](https://discuss.elastic.co/u/Jason_Wee)\ **Post date:** [December 19, 2013, 9:50am UTC](https://discuss.elastic.co/t/shard-relocation-progress/14940/2 "2013-12-19T09:50:40Z")
Hi, comment inline. On Thu, Dec 19, 2013 at 5:28 AM, Mohit Anchlia [mohitanchlia@gmail.com](mailto:mohitanchlia@gmail.com)wrote: > What's the best way to monitor shard relocation that occurrs when one add > new nodes? Turn on the logging and tailing the logs and start reading, but if you want visual, you can use elasticsearch-head > Is there a way to control the relocation and do it manually with few > shards at a time? Yes, you should spend sometime to read on the cluster settings, > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. in there, you should find the parameter you want. Try experiment those settings in your test environment. > What are the best practices for cluster that is contantly received high > volume traffic? throttle them? > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAOT3TWovrNe4cJvqAuvPV26zErezm0eqo9rF%2B%3DFpFk\_Pzh-q0Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAOT3TWovrNe4cJvqAuvPV26zErezm0eqo9rF%2B%3DFpFk_Pzh-q0Q%40mail.gmail.com) > . > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAHO4itznn%2BG4D2s5SNNC3QARObz8j1gj6uuJ2numKngZukxuEA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHO4itznn%2BG4D2s5SNNC3QARObz8j1gj6uuJ2numKngZukxuEA%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [December 19, 2013, 3:13pm UTC](https://discuss.elastic.co/t/shard-relocation-progress/14940/3 "2013-12-19T15:13:11Z")
Hey, Monitor: it is all in the cluster state. Use a plugin like head or paramedic to monitor it. If you play around with 1.0.0 beta, check out the Cat API, which is a neat commandline interface to find out, what happens in your cluster. Control: You can control relocation. You can enable/disable it, set up rules which nodes should hold which shards or if shards should be moved around based on disk space allocation (allocation, awareness, forced awareness). Practices: You can throttle the amount of concurrent relocations (per node), you can throttle its throughput to make sure that daily business like searching and indexing still work as in lines with your SLA. Hope this helps is there any specific hidden concrete question? 🙂 --Alex On Wed, Dec 18, 2013 at 10:28 PM, Mohit Anchlia [mohitanchlia@gmail.com](mailto:mohitanchlia@gmail.com)wrote: > What's the best way to monitor shard relocation that occurrs when one add > new nodes? > > Is there a way to control the relocation and do it manually with few > shards at a time? > > What are the best practices for cluster that is contantly received high > volume traffic? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAOT3TWovrNe4cJvqAuvPV26zErezm0eqo9rF%2B%3DFpFk\_Pzh-q0Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAOT3TWovrNe4cJvqAuvPV26zErezm0eqo9rF%2B%3DFpFk_Pzh-q0Q%40mail.gmail.com) > . > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGCwEM\_kR7xyCZUMidEHdpHxOZV9hPfa2TXgZ7y5mzXM8jebFA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGCwEM_kR7xyCZUMidEHdpHxOZV9hPfa2TXgZ7y5mzXM8jebFA%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Mohit\_Anchlia](https://avatars.discourse-cdn.com/v4/letter/m/848f3c/32.png) [@Mohit\_Anchlia](https://discuss.elastic.co/u/Mohit_Anchlia)\ **Post date:** [December 19, 2013, 5:38pm UTC](https://discuss.elastic.co/t/shard-relocation-progress/14940/4 "2013-12-19T17:38:36Z")
Thanks! that helps, I'll look at these settings. Regarding monitoring is there a way to look at the progress. On Thu, Dec 19, 2013 at 7:13 AM, Alexander Reelsen [alr@spinscale.de](mailto:alr@spinscale.de) wrote: > Hey, > > Monitor: it is all in the cluster state. Use a plugin like head or > paramedic to monitor it. If you play around with 1.0.0 beta, check out the > Cat API, which is a neat commandline interface to find out, what happens in > your cluster. > > Control: You can control relocation. You can enable/disable it, set up > rules which nodes should hold which shards or if shards should be moved > around based on disk space allocation (allocation, awareness, forced > awareness). > > Practices: You can throttle the amount of concurrent relocations (per > node), you can throttle its throughput to make sure that daily business > like searching and indexing still work as in lines with your SLA. > > Hope this helps is there any specific hidden concrete question? 🙂 > > --Alex > > On Wed, Dec 18, 2013 at 10:28 PM, Mohit Anchlia [mohitanchlia@gmail.com](mailto:mohitanchlia@gmail.com)wrote: > > > What's the best way to monitor shard relocation that occurrs when one add > > new nodes? > > > > Is there a way to control the relocation and do it manually with few > > shards at a time? > > > > What are the best practices for cluster that is contantly received high > > volume traffic? > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAOT3TWovrNe4cJvqAuvPV26zErezm0eqo9rF%2B%3DFpFk\_Pzh-q0Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAOT3TWovrNe4cJvqAuvPV26zErezm0eqo9rF%2B%3DFpFk_Pzh-q0Q%40mail.gmail.com) > > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAGCwEM\_kR7xyCZUMidEHdpHxOZV9hPfa2TXgZ7y5mzXM8jebFA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGCwEM_kR7xyCZUMidEHdpHxOZV9hPfa2TXgZ7y5mzXM8jebFA%40mail.gmail.com) > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAOT3TWr11LH-e\_Z-iNp%3D15a9Y1PJhpOBUM%3DpRf4jmPEMgtuMqg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAOT3TWr11LH-e_Z-iNp%3D15a9Y1PJhpOBUM%3DpRf4jmPEMgtuMqg%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [December 30, 2013, 7:04am UTC](https://discuss.elastic.co/t/shard-relocation-progress/14940/5 "2013-12-30T07:04:15Z")
Hi Mohit, On Thursday, December 19, 2013 12:38:36 PM UTC-5, Mo wrote: > Thanks! that helps, I'll look at these settings. > > Regarding monitoring is there a way to look at the progress. Of the migration state/%/ETA for a single shard? I don't think there's anything that will show you that.... But if you have N shards moving around and want to see "progress over time" in the sense of being able to see the number of shards moving around currently and at different points in time/history, you can see that in SPM. .... like in this image, more or less: [http://elasticsearch-users.115913.n3.nabble.com/attachment/3803285/0/spm-es-shard-dance.png](http://elasticsearch-users.115913.n3.nabble.com/attachment/3803285/0/spm-es-shard-dance.png) ## Otis Performance Monitoring \* Log Analytics \* Search Analytics Solr & Elasticsearch Support \* [http://sematext.com/](http://sematext.com/) > On Thu, Dec 19, 2013 at 7:13 AM, Alexander Reelsen \<[a...@spinscale.de](mailto:a...@spinscale.de)\ > > > wrote: > > > Hey, > > > > Monitor: it is all in the cluster state. Use a plugin like head or > > paramedic to monitor it. If you play around with 1.0.0 beta, check out the > > Cat API, which is a neat commandline interface to find out, what happens in > > your cluster. > > > > Control: You can control relocation. You can enable/disable it, set up > > rules which nodes should hold which shards or if shards should be moved > > around based on disk space allocation (allocation, awareness, forced > > awareness). > > > > Practices: You can throttle the amount of concurrent relocations (per > > node), you can throttle its throughput to make sure that daily business > > like searching and indexing still work as in lines with your SLA. > > > > Hope this helps is there any specific hidden concrete question? 🙂 > > > > --Alex > > > > On Wed, Dec 18, 2013 at 10:28 PM, Mohit Anchlia \<[mohita...@gmail.com](mailto:mohita...@gmail.com)\ > > > > > wrote: > > > > > What's the best way to monitor shard relocation that occurrs when one > > > add new nodes? > > > > > > Is there a way to control the relocation and do it manually with few > > > shards at a time? > > > > > > What are the best practices for cluster that is contantly received high > > > volume traffic? > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAOT3TWovrNe4cJvqAuvPV26zErezm0eqo9rF%2B%3DFpFk\_Pzh-q0Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAOT3TWovrNe4cJvqAuvPV26zErezm0eqo9rF%2B%3DFpFk_Pzh-q0Q%40mail.gmail.com) > > > . > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAGCwEM\_kR7xyCZUMidEHdpHxOZV9hPfa2TXgZ7y5mzXM8jebFA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGCwEM_kR7xyCZUMidEHdpHxOZV9hPfa2TXgZ7y5mzXM8jebFA%40mail.gmail.com) > > . > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/97f6a19e-7dd8-4b0d-994e-6ae0199d61c9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/97f6a19e-7dd8-4b0d-994e-6ae0199d61c9%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:59am UTC](https://discuss.elastic.co/t/shard-relocation-progress/14940/6 "2017-07-06T01:59:06Z")