Hi Community,
I have a few questions regarding a DC–DR setup for Elasticsearch, Kibana, and Logstash. Your insights would be greatly appreciated.
We plan to configure Hot and Warm nodes in the primary DC. Can the same configuration be replicated in the DR site with continuous data synchronization? If so, is it possible to synchronize the tiers independently—for example, DC Hot node data syncing only to the DR Hot node, and Warm node data syncing only to the DR Warm node?
Can Elasticsearch nodes be managed through a load balancer across both the DC and DR sites?
We are considering deploying Kibana in a DC–DR or high-availability setup using a load balancer. In the event of an Elasticsearch node failure in the DC, can Kibana in the DC fail over to Elasticsearch nodes in the DR site? Is this a viable approach?
Is there a recommended way to implement a DC–DR or high-availability setup for Logstash? Since applications send logs directly to Logstash, we want to ensure log ingestion and monitoring remain uninterrupted.
Can Hot and Warm nodes be installed on the same server using separate partitions?
It would help if you could tell us a bit about the use case and requirements. Is this a logging and/or metrics use case given that you are considering a hot-warm architecture? Is your data immutable?
What led you to decide you need a hot-warm architecture?
What are the data volumes for the use case? How much data do you intent to ingest every day? What is the required retention period for this data?
What are the requirements that dictate a DC-DR setup? How far apart are the data centres and what is the connectivity and latency between them?
Typically the other nodes of the cluster within the DC would seamlessly take over, so failover to DR site would generally not be required for this.