# How to migrate index without disturbing client? **URL:** \ **Category:** Elasticsearch\ **Created:** [May 29, 2015, 12:38pm UTC](https://discuss.elastic.co/t/how-to-migrate-index-without-disturbing-client/23722 "2015-05-29T12:38:24Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Mehmet\_Ozer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mehmet_ozer/32/81567_2.png) [@Mehmet\_Ozer](https://discuss.elastic.co/u/Mehmet_Ozer)\ **Post date:** [May 29, 2015, 12:38pm UTC](https://discuss.elastic.co/t/how-to-migrate-index-without-disturbing-client/23722/1 "2015-05-29T12:38:24Z")
Hello, I am running a cluster which contains many indexes. In the near future I will have new machines so that I will need to migrate my indexes to those machines. I have thought some scenarios but it will not be possible to do so. Let me explain what I thought and why it was not possible; - I wanted to put the new machines into existing cluster and then I thought I could use exclude.\_ip option to exclude existing machines. So that old indexes would have migrated. This seemed possible for me but I can not stop working on the index, users should continue making searches on the indexes. So can you please tell me how I can migrate the data without disturbing clients ? Thank you ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [May 29, 2015, 12:44pm UTC](https://discuss.elastic.co/t/how-to-migrate-index-without-disturbing-client/23722/2 "2015-05-29T12:44:52Z")
What you just described should work fine. exclude.\_ip will move the shards off of the nodes you exclude but queries and updates can proceed while this is happening because the data is still on the old nodes. The updates will make their way to the new copies via a transaction log reply mechanism. Depending on how the clients connect you might need to shift their connection uris to the new servers and that might require restarting them - depending on how your clients work. On Fri, May 29, 2015 at 8:38 AM, mehmet özer [mehmetozer1988@gmail.com](mailto:mehmetozer1988@gmail.com) wrote: > Hello, > > I am running a cluster which contains many indexes. In the near future I > will have new machines so that I will need to migrate my indexes to those > machines. I have thought some scenarios but it will not be possible to do > so. Let me explain what I thought and why it was not possible; > > - I wanted to put the new machines into existing cluster and then I > thought I could use exclude.\_ip option to exclude existing machines. So > that old indexes would have migrated. > > This seemed possible for me but I can not stop working on the index, users > should continue making searches on the indexes. So can you please tell me > how I can migrate the data without disturbing clients ? > > Thank you > > ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) > > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1mYXppwguwnPCfyMCdG4jx0AmQBCMoqcxDE%2Bb9U1\_nmQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1mYXppwguwnPCfyMCdG4jx0AmQBCMoqcxDE%2Bb9U1_nmQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Mehmet\_Ozer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mehmet_ozer/32/81567_2.png) [@Mehmet\_Ozer](https://discuss.elastic.co/u/Mehmet_Ozer)\ **Post date:** [May 29, 2015, 12:51pm UTC](https://discuss.elastic.co/t/how-to-migrate-index-without-disturbing-client/23722/3 "2015-05-29T12:51:11Z")
Thanks for your answer. I didnt know the exclude doesnt disable the research. But according to the restart, - I cannot restart the cluster because I have 9 tera of data so it can take one day to restart all cluster. In addition to that old machines and new machines will not know each other( I mean unicast variable of the elasticsearch.yml, the old machines will know just old ones). Do you think it can cause a problem ? Thanks 29 Mayıs 2015 Cuma 14:45:40 UTC+2 tarihinde Nikolas Everett yazdı: > What you just described should work fine. exclude.\_ip will move the shards > off of the nodes you exclude but queries and updates can proceed while this > is happening because the data is still on the old nodes. The updates will > make their way to the new copies via a transaction log reply mechanism. > > Depending on how the clients connect you might need to shift their > connection uris to the new servers and that might require restarting them - > depending on how your clients work. > > On Fri, May 29, 2015 at 8:38 AM, mehmet özer \<[mehmeto...@gmail.com](mailto:mehmeto...@gmail.com) > \\> wrote: > > > Hello, > > > > I am running a cluster which contains many indexes. In the near future I > > will have new machines so that I will need to migrate my indexes to those > > machines. I have thought some scenarios but it will not be possible to do > > so. Let me explain what I thought and why it was not possible; > > > > - I wanted to put the new machines into existing cluster and then I > > thought I could use exclude.\_ip option to exclude existing machines. So > > that old indexes would have migrated. > > > > This seemed possible for me but I can not stop working on the index, > > users should continue making searches on the indexes. So can you please > > tell me how I can migrate the data without disturbing clients ? > > > > Thank you > > > > ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) > > > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c8cc4028-9ca6-4ffa-ad92-d36082062839%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c8cc4028-9ca6-4ffa-ad92-d36082062839%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [May 29, 2015, 1:08pm UTC](https://discuss.elastic.co/t/how-to-migrate-index-without-disturbing-client/23722/4 "2015-05-29T13:08:50Z")
You certainly will have to think of a way to add new cluster masters - but you can still do it with unicast discovery I think. Yeah - exclude doesn't disable searches - it just causes elasticsearch to move the shards using it normal shard moving mechanism. Its really just adding a routing rule. You shouldn't have to restart the whole cluster - but you will have to convince your clients to look at the new nodes. The last time I did a full restart of the Elasticsearch cluster was about a year ago when I upgraded from 0.90.something to 1.1.something and I expect the next full restart to be the upgrade to 2.0 when its released. In the mean time we've: 1. Done about twenty rolling restarts to pick up new code 2. Replaced all the hard drives in all the nodes - two at a time - moving data off of the nodes we were restarting using excluse.\_ip 3. Changed mount options on the elasticsearch data partition using the same technique 4. More than doubled the number of nodes in the cluster. In our case the application connects using linux virtual server so all we had to do is teach the application to retry on connection errors for when lvs round-robin-ed it to a down Elasticsearch node and teach our lvs pool maintainer to depool elasticsearch nodes that are down. I'm not an expert on the Java client so I don't know what you'd have to do to update and java applications to the new nodes. If you are using a proxy like nginx you'd just have to change where it points. Nik On Fri, May 29, 2015 at 8:51 AM, mehmet özer [mehmetozer1988@gmail.com](mailto:mehmetozer1988@gmail.com) wrote: > Thanks for your answer. I didnt know the exclude doesnt disable the > research. But according to the restart, > > - I cannot restart the cluster because I have 9 tera of data so it can > take one day to restart all cluster. In addition to that old machines and > new machines will not know each other( I mean unicast variable of the > elasticsearch.yml, the old machines will know just old ones). Do you think > it can cause a problem ? > > Thanks > > 29 Mayıs 2015 Cuma 14:45:40 UTC+2 tarihinde Nikolas Everett yazdı: > > > What you just described should work fine. exclude.\_ip will move the > > shards off of the nodes you exclude but queries and updates can proceed > > while this is happening because the data is still on the old nodes. The > > updates will make their way to the new copies via a transaction log reply > > mechanism. > > > > Depending on how the clients connect you might need to shift their > > connection uris to the new servers and that might require restarting them - > > depending on how your clients work. > > > > On Fri, May 29, 2015 at 8:38 AM, mehmet özer [mehmeto...@gmail.com](mailto:mehmeto...@gmail.com) > > wrote: > > > > > Hello, > > > > > > I am running a cluster which contains many indexes. In the near future I > > > will have new machines so that I will need to migrate my indexes to those > > > machines. I have thought some scenarios but it will not be possible to do > > > so. Let me explain what I thought and why it was not possible; > > > > > > - I wanted to put the new machines into existing cluster and then I > > > thought I could use exclude.\_ip option to exclude existing machines. So > > > that old indexes would have migrated. > > > > > > This seemed possible for me but I can not stop working on the index, > > > users should continue making searches on the indexes. So can you please > > > tell me how I can migrate the data without disturbing clients ? > > > > > > Thank you > > > > > > ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) > > > > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) > > * * * > > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/c8cc4028-9ca6-4ffa-ad92-d36082062839%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c8cc4028-9ca6-4ffa-ad92-d36082062839%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/c8cc4028-9ca6-4ffa-ad92-d36082062839%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/c8cc4028-9ca6-4ffa-ad92-d36082062839%40googlegroups.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd0JOF3%2BAQ4Y0Pfsp8yWheec6WwnrKEuqjCPN7rPHqH24g%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd0JOF3%2BAQ4Y0Pfsp8yWheec6WwnrKEuqjCPN7rPHqH24g%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Mehmet\_Ozer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mehmet_ozer/32/81567_2.png) [@Mehmet\_Ozer](https://discuss.elastic.co/u/Mehmet_Ozer)\ **Post date:** [May 29, 2015, 1:39pm UTC](https://discuss.elastic.co/t/how-to-migrate-index-without-disturbing-client/23722/5 "2015-05-29T13:39:51Z")
Thats great. Thanks for your help ! 29 Mayıs 2015 Cuma 15:09:21 UTC+2 tarihinde Nikolas Everett yazdı: > You certainly will have to think of a way to add new cluster masters - but > you can still do it with unicast discovery I think. > > Yeah - exclude doesn't disable searches - it just causes elasticsearch to > move the shards using it normal shard moving mechanism. Its really just > adding a routing rule. > > You shouldn't have to restart the whole cluster - but you will have to > convince your clients to look at the new nodes. > > The last time I did a full restart of the Elasticsearch cluster was about > a year ago when I upgraded from 0.90.something to 1.1.something and I > expect the next full restart to be the upgrade to 2.0 when its released. In > the mean time we've: > > 1. Done about twenty rolling restarts to pick up new code > 2. Replaced all the hard drives in all the nodes - two at a time - moving > data off of the nodes we were restarting using excluse.\_ip > 3. Changed mount options on the elasticsearch data partition using the > same technique > 4. More than doubled the number of nodes in the cluster. > > In our case the application connects using linux virtual server so all we > had to do is teach the application to retry on connection errors for when > lvs round-robin-ed it to a down Elasticsearch node and teach our lvs pool > maintainer to depool elasticsearch nodes that are down. > > I'm not an expert on the Java client so I don't know what you'd have to do > to update and java applications to the new nodes. If you are using a proxy > like nginx you'd just have to change where it points. > > Nik > > On Fri, May 29, 2015 at 8:51 AM, mehmet özer \<[mehmeto...@gmail.com](mailto:mehmeto...@gmail.com) > \\> wrote: > > > Thanks for your answer. I didnt know the exclude doesnt disable the > > research. But according to the restart, > > > > - I cannot restart the cluster because I have 9 tera of data so it can > > take one day to restart all cluster. In addition to that old machines and > > new machines will not know each other( I mean unicast variable of the > > elasticsearch.yml, the old machines will know just old ones). Do you think > > it can cause a problem ? > > > > Thanks > > > > 29 Mayıs 2015 Cuma 14:45:40 UTC+2 tarihinde Nikolas Everett yazdı: > > > > > What you just described should work fine. exclude.\_ip will move the > > > shards off of the nodes you exclude but queries and updates can proceed > > > while this is happening because the data is still on the old nodes. The > > > updates will make their way to the new copies via a transaction log reply > > > mechanism. > > > > > > Depending on how the clients connect you might need to shift their > > > connection uris to the new servers and that might require restarting them - > > > depending on how your clients work. > > > > > > On Fri, May 29, 2015 at 8:38 AM, mehmet özer [mehmeto...@gmail.com](mailto:mehmeto...@gmail.com) > > > wrote: > > > > > > > Hello, > > > > > > > > I am running a cluster which contains many indexes. In the near future > > > > I will have new machines so that I will need to migrate my indexes to those > > > > machines. I have thought some scenarios but it will not be possible to do > > > > so. Let me explain what I thought and why it was not possible; > > > > > > > > - I wanted to put the new machines into existing cluster and then I > > > > thought I could use exclude.\_ip option to exclude existing machines. So > > > > that old indexes would have migrated. > > > > > > > > This seemed possible for me but I can not stop working on the index, > > > > users should continue making searches on the indexes. So can you please > > > > tell me how I can migrate the data without disturbing clients ? > > > > > > > > Thank you > > > > > > > > ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) > > > > > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/3df7a8b2-c231-4b27-b072-c4c8e790cd3c%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > Please update your bookmarks! We have moved to > > > [https://discuss.elastic.co/](https://discuss.elastic.co/) > > > > * * * > > > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/c8cc4028-9ca6-4ffa-ad92-d36082062839%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c8cc4028-9ca6-4ffa-ad92-d36082062839%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/c8cc4028-9ca6-4ffa-ad92-d36082062839%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/c8cc4028-9ca6-4ffa-ad92-d36082062839%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0f797fb5-e8ef-4a34-9513-6ae891435976%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0f797fb5-e8ef-4a34-9513-6ae891435976%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:10am UTC](https://discuss.elastic.co/t/how-to-migrate-index-without-disturbing-client/23722/6 "2017-07-06T00:10:49Z")