# Index Migration Strategy

**URL:** <https://discuss.elastic.co/t/index-migration-strategy/66435>\
**Category:** Elasticsearch\
**Created:** [November 17, 2016, 5:29pm UTC](https://discuss.elastic.co/t/index-migration-strategy/66435 "2016-11-17T17:29:10Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Paul\_Ainslie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paul_ainslie/32/55031_2.png) [@Paul\_Ainslie](https://discuss.elastic.co/u/Paul_Ainslie)\
**Post date:** [November 17, 2016, 5:29pm UTC](https://discuss.elastic.co/t/index-migration-strategy/66435/1 "2016-11-17T17:29:10Z")

</div>

Hi, I'm wondering if there's a way to migrate to a new ES index without shutting down logstash on our web servers.

I'm using Logstash (ES to ES configuration) to migrate data to a new index. I normally use the reindex function but this time I needed the `useragent` filter. Anyway, in my experience the only way to migrate to a new index **without loosing any docs** is to shut down logstash on each of our nginx servers. Thus the source/old index is not in a state of flux.

Here's what I do:  
a) shut down logstash (logfile to ES config) on all our web servers  
b) start logstash (ES to ES config) on my ES cluster which migrates to the new index  
c) when migration is complete, shut down logstash (ES to ES config)  
d) point my alias to the new index  
e) start logstash (points to the alias) on all our web servers

This works well, but the problem with this strategy is that logstash is down for a few hours. Is there anyway around this?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [November 21, 2016, 2:39am UTC](https://discuss.elastic.co/t/index-migration-strategy/66435/2 "2016-11-21T02:39:48Z")

</div>

If you were using a broker (kafka, redis etc) you could leverage that, but in this case you can't work around the need to shut things down ☹

Can you explain a little more about the index structure you are using, specifically how the alias is setup.

---

<div class="post-metadata">

**Author:** ![Paul\_Ainslie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paul_ainslie/32/55031_2.png) [@Paul\_Ainslie](https://discuss.elastic.co/u/Paul_Ainslie)\
**Post date:** [November 21, 2016, 3:33am UTC](https://discuss.elastic.co/t/index-migration-strategy/66435/3 "2016-11-21T03:33:38Z")

</div>

Thanks Mark;

It's simple in that one alias points to one index. Logstash forwards to the alias therefore I don't need to update that.

In the short term I've been using Ansible to shut down/startup logstash on all nginx servers before/after a migration.

I'll look into using a broker as you mentioned, that sounds like a good long term plan.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [November 21, 2016, 5:24am UTC](https://discuss.elastic.co/t/index-migration-strategy/66435/4 "2016-11-21T05:24:54Z")

</div>

Are you using time based indices under the hood?

---

<div class="post-metadata">

**Author:** ![Paul\_Ainslie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paul_ainslie/32/55031_2.png) [@Paul\_Ainslie](https://discuss.elastic.co/u/Paul_Ainslie)\
**Post date:** [November 25, 2016, 8:10pm UTC](https://discuss.elastic.co/t/index-migration-strategy/66435/5 "2016-11-25T20:10:54Z")

</div>

Nope, no time based indices.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 23, 2016, 8:11pm UTC](https://discuss.elastic.co/t/index-migration-strategy/66435/6 "2016-12-23T20:11:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
