I have a problem with Elasticsearch.
At present we have one elasticsearch for all environment.
That Elasticsearch is connected with Logstash (For logs indexing purpose) as well as Zipkin (Logs Streaming Purpose).
Here the problem I am getting is if the Elasticsearch went down, logs data from Zipkin are not able to stream and losing the data.
If the Elasticsearch is down from date 01/Apr/2018 - 04/Apr/2018 and got up on 05/Apr/2018.
The streamed data from 01 - 04 is not able to stream in Kibana and losing data up to 4th.
So I want to make Elasticsearch as high availability.
Please go through the requirement drawn in the image.
In the above diagram the Zipkin and Logstash indices are pointing to Elasticsearch 1 and output from Elasticsearch 1 is displaying in Kibana.
If the Elasticsearch 1 is down both Zipkin and Logstash have to point Elasticsearch 2 and new streaming data from Zipkin and new indices from Logstash are have to be available with the data of Elasticsearch 1 in Kibana.(That means both Elasticsearch 1 and Elasticsearch 2 have to be interconnected).
Currently we have only one Elasticsearch.
This is an high priority requirement.
Please help me with the solutions on how to configure more Elasticsearch's, how to make them interlinked and how to make them as high available.
Thanks in advance.