How to make Elasticsearch as high available?

Elastic Stack Elasticsearch
#1

I have a problem with Elasticsearch.

At present we have one elasticsearch for all environment.

That Elasticsearch is connected with Logstash (For logs indexing purpose) as well as Zipkin (Logs Streaming Purpose).

Here the problem I am getting is if the Elasticsearch went down, logs data from Zipkin are not able to stream and losing the data.

For Example:
If the Elasticsearch is down from date 01/Apr/2018 - 04/Apr/2018 and got up on 05/Apr/2018.
The streamed data from 01 - 04 is not able to stream in Kibana and losing data up to 4th.

So I want to make Elasticsearch as high availability.

For Example:
Please go through the requirement drawn in the image.

Elasticsearch%20high%20availability%20Diagram

In the above diagram the Zipkin and Logstash indices are pointing to Elasticsearch 1 and output from Elasticsearch 1 is displaying in Kibana.
If the Elasticsearch 1 is down both Zipkin and Logstash have to point Elasticsearch 2 and new streaming data from Zipkin and new indices from Logstash are have to be available with the data of Elasticsearch 1 in Kibana.(That means both Elasticsearch 1 and Elasticsearch 2 have to be interconnected).
Currently we have only one Elasticsearch.

This is an high priority requirement.

Please help me with the solutions on how to configure more Elasticsearch's, how to make them interlinked and how to make them as high available.

Thanks in advance.

#2

In logstash elasticsearch output plugin you can specify an array of nodes.
Just put your other nodes there.

#3

@dadoonet
If i do it in the logstash level it won't be available for zipkin. From Zipkin we are directly sending the stream to Elasticsearch. So we have to configure Elasticsearch as common usable for both Zipkin and Logstash.

#4

Does Zipkin supports multiple elasticsearch instances as Logstash does?

#5

@dadoonet
Yes...

#6

So. What is the problem then? I don't understand.

Just start 2 (better 3) elasticsearch instances, configure them in LS and Zipkin and then you're done, right?

#7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.