our plan is to store logging information in three datacenters. The data should be high available - also in case of a complete outage of one datacenter.
A tribe node is not an option as in case of a datacenter outage parts of the data won't be available.
Our current architechture is: Logstash (Shipper) -> Redis (Broker - no cluster) -> Logstash (Indexer) -> ES.
Our idea is to copy the data to all datacenters on Broker level. We think about using the cluster functionality of Redis. Maybe Kafka or RabbitMQ are also suitable instead of Redis.
I'd suggest you start with checking for other threads as this has been asked a few times before.
If you have specific questions afterwards then we'll be here to help!
I already checked other threads but they weren't really helpfull. Some use backup & restore which is not a really good solution. I didn't find articles where an already experienced solution is explained. But I will search again. If you know an article it would be nice if you could send me a link.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
