# Moving whole old index **URL:** \ **Category:** Elasticsearch\ **Created:** [October 21, 2013, 2:23pm UTC](https://discuss.elastic.co/t/moving-whole-old-index/14040 "2013-10-21T14:23:22Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Krager](https://avatars.discourse-cdn.com/v4/letter/k/f07891/32.png) [@Krager](https://discuss.elastic.co/u/Krager)\ **Post date:** [October 21, 2013, 2:23pm UTC](https://discuss.elastic.co/t/moving-whole-old-index/14040/1 "2013-10-21T14:23:22Z")
Hi everyone! I got a 4 nodes cluster, 3 of them have 100Gb SSD and a other have a 4 Tb HDD. Each indexes has 6 shards. Since every daily index is using around 50Gb I cannot store more than 6 days. I would like to store old indexes to the 4 Tb HDD. Is there a good way to move the old shards/indexes automatically, like with a cron script? Thanks. Cheers, Krager -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Zachary\_Lammers](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zachary_lammers/32/1651_2.png) [@Zachary\_Lammers](https://discuss.elastic.co/u/Zachary_Lammers)\ **Post date:** [October 21, 2013, 2:46pm UTC](https://discuss.elastic.co/t/moving-whole-old-index/14040/2 "2013-10-21T14:46:45Z")
I'd love to know this as well, as I have servers with both SSD and HDD arrays where i'd like to store the most recent days data on SSD, then migrate to HDDs. Note: I have not tested this yet, but this was my 'plan of attack' to see if I could do this: I thought to use rerouting: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-reroute.html) - which looks like it can be easily enough scripted to force index/shard movement to a slower array. But it looks like the cluster will re-balance after any forced re-route. So, I was going to try using index shard allocation config: ( [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-allocation.html)) to exclude my 'slow' node, and then only force it on manual re-route above. -Zachary On Monday, October 21, 2013 9:23:22 AM UTC-5, Krager wrote: > Hi everyone! > > I got a 4 nodes cluster, 3 of them have 100Gb SSD and a other have a 4 Tb > HDD. > Each indexes has 6 shards. > Since every daily index is using around 50Gb I cannot store more than 6 > days. > > I would like to store old indexes to the 4 Tb HDD. > > Is there a good way to move the old shards/indexes automatically, like > with a cron script? > > Thanks. > > Cheers, > Krager -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Krager](https://avatars.discourse-cdn.com/v4/letter/k/f07891/32.png) [@Krager](https://discuss.elastic.co/u/Krager)\ **Post date:** [October 21, 2013, 2:58pm UTC](https://discuss.elastic.co/t/moving-whole-old-index/14040/3 "2013-10-21T14:58:06Z")
That's the direction I was going to. But I would have excluded SSD nodes and required the HDD when index gets too old, then doing a reroute so that it won't be rebalanced after. But reroute could be a pain as you've got to specify each shard and from which node it is coming from. I haven't tested this still, and I wonder if there is a way to move the whole index or moving each shards wherever they're coming from. --Krager Le lundi 21 octobre 2013 16:46:45 UTC+2, Zachary Lammers a écrit : > I'd love to know this as well, as I have servers with both SSD and HDD > arrays where i'd like to store the most recent days data on SSD, then > migrate to HDDs. > > Note: I have not tested this yet, but this was my 'plan of attack' to see > if I could do this: > > I thought to use rerouting: > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-reroute.html) - > which looks like it can be easily enough scripted to force index/shard > movement to a slower array. But it looks like the cluster will re-balance > after any forced re-route. > > So, I was going to try using index shard allocation config: ( > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-allocation.html)) > to exclude my 'slow' node, and then only force it on manual re-route above. > > -Zachary > > On Monday, October 21, 2013 9:23:22 AM UTC-5, Krager wrote: > > > Hi everyone! > > > > I got a 4 nodes cluster, 3 of them have 100Gb SSD and a other have a 4 Tb > > HDD. > > Each indexes has 6 shards. > > Since every daily index is using around 50Gb I cannot store more than 6 > > days. > > > > I would like to store old indexes to the 4 Tb HDD. > > > > Is there a good way to move the old shards/indexes automatically, like > > with a cron script? > > > > Thanks. > > > > Cheers, > > Krager -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Krager](https://avatars.discourse-cdn.com/v4/letter/k/f07891/32.png) [@Krager](https://discuss.elastic.co/u/Krager)\ **Post date:** [October 22, 2013, 3:05pm UTC](https://discuss.elastic.co/t/moving-whole-old-index/14040/4 "2013-10-22T15:05:44Z")
Ok so, it seems there is no need, to reroute. I just did an exclude for SSD and include on HDD and the whole index moved automatically (on live cluster). The index doesn't seems to be rerouted so far it looks just good for now. Here is the command: curl -XPUT localhost:9200/oldindex/\_settings -d '{ "index.routing.allocation.exclude.\_ip" : "X.X.X.X,Y.Y.Y.Y", "index.routing.allocation.include.\_ip" : "Z.Z.Z.Z" }' Of course, oldindex is a random index name and X.X.X.X stands for the IP address of the node Cheers, Krager -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Zachary\_Lammers](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zachary_lammers/32/1651_2.png) [@Zachary\_Lammers](https://discuss.elastic.co/u/Zachary_Lammers)\ **Post date:** [November 5, 2013, 2:32pm UTC](https://discuss.elastic.co/t/moving-whole-old-index/14040/5 "2013-11-05T14:32:45Z")
I was able to do my test as well, almost same command as below, but I did it with tags -- on my nodes, i have a "node.tag: [current|archive]", and my template excludes my 'archive' tag in the settings ( "index.routing.allocation.exclude.tag" : "archive" ) Then, to move my index, I ran the very similar command below. I couldn't use IPs like you did, as I have large SSD arrays and HDD arrays on the same server, so I run a 'current store' ES node and an 'archive' ES node at the same time. curl -XPUT localhost:9200//\_settings -d '{ "index.routing.allocation.exclude.tag" : "current", "index.routing.allocation.include.tag" : "archive" }' On Tuesday, October 22, 2013 10:05:44 AM UTC-5, Krager wrote: > Ok so, it seems there is no need, to reroute. > > I just did an exclude for SSD and include on HDD and the whole index moved > automatically (on live cluster). > > The index doesn't seems to be rerouted so far it looks just good for now. > Here is the command: > curl -XPUT localhost:9200/oldindex/\_settings -d '{ > "index.routing.allocation.exclude.\_ip" : "X.X.X.X,Y.Y.Y.Y", > "index.routing.allocation.include.\_ip" : "Z.Z.Z.Z" > }' > > Of course, oldindex is a random index name and X.X.X.X stands for the IP > address of the node > > Cheers, > > Krager -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:08am UTC](https://discuss.elastic.co/t/moving-whole-old-index/14040/6 "2017-07-06T02:08:51Z")