Hi, I'm wondering if there's a way to migrate to a new ES index without shutting down logstash on our web servers.
I'm using Logstash (ES to ES configuration) to migrate data to a new index. I normally use the reindex function but this time I needed the
useragent filter. Anyway, in my experience the only way to migrate to a new index without loosing any docs is to shut down logstash on each of our nginx servers. Thus the source/old index is not in a state of flux.
Here's what I do:
a) shut down logstash (logfile to ES config) on all our web servers
b) start logstash (ES to ES config) on my ES cluster which migrates to the new index
c) when migration is complete, shut down logstash (ES to ES config)
d) point my alias to the new index
e) start logstash (points to the alias) on all our web servers
This works well, but the problem with this strategy is that logstash is down for a few hours. Is there anyway around this?