Planned total network page - how will the cluster react?

Elastic Stack Elasticsearch
1

We have a planned network outage for 8 hours that will stop communication between all hosts, including between nodes in our elastic cluster.

We have 3 master nodes and 2 minimum master nodes set. Using logstash to write to the cluster and apps read from it. Indexes are replicated with 1p, 2r.

I don't want to have to organise for the cluster to be shut down and restarted.

If I do nothing, will the cluster have any problems? I didn't think it would, apart from maybe lots of errors being logged.

But should I be cautious, and stop the logstash jobs, and maybe set shard allocation to a long period to cover the outage? Or something else? Not really sure what is best.

2

What version are you using?

The cluster won't work while the network is broken but it should recover on its own when connectivity is restored. You don't need to do anything special to make that happen, but I suppose it might help it recover a bit quicker if you set cluster.routing.allocation.enable: none beforehand and then remove this setting once the outage is over.

I don't know how Logstash will react to the cluster being unavailable and/or rejecting its writes. Best to ask that again in the Logstash forum.

3

Thanks very much for your reply David. We're on version 6.2.3.

4

Tangential, but please upgrade as 6.2 reached EOL 2 years ago.

5

Yes, that's definitely something we need to do, getting such work prioritised is difficult, even though the longer we wait the harder it will get.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.