Hi Community,
I have a few questions regarding a DC–DR setup for Elasticsearch, Kibana, and Logstash. Your insights would be greatly appreciated.
We plan to configure Hot and Warm nodes in the primary DC. Can the same configuration be replicated in the DR site with continuous data synchronization? If so, is it possible to synchronize the tiers independently—for example, DC Hot node data syncing only to the DR Hot node, and Warm node data syncing only to the DR Warm node?
Can Elasticsearch nodes be managed through a load balancer across both the DC and DR sites?
We are considering deploying Kibana in a DC–DR or high-availability setup using a load balancer. In the event of an Elasticsearch node failure in the DC, can Kibana in the DC fail over to Elasticsearch nodes in the DR site? Is this a viable approach?
Is there a recommended way to implement a DC–DR or high-availability setup for Logstash? Since applications send logs directly to Logstash, we want to ensure log ingestion and monitoring remain uninterrupted.
Can Hot and Warm nodes be installed on the same server using separate partitions?
It would help if you could tell us a bit about the use case and requirements. Is this a logging and/or metrics use case given that you are considering a hot-warm architecture? Is your data immutable?
What led you to decide you need a hot-warm architecture?
What are the data volumes for the use case? How much data do you intent to ingest every day? What is the required retention period for this data?
What are the requirements that dictate a DC-DR setup? How far apart are the data centres and what is the connectivity and latency between them?
Typically the other nodes of the cluster within the DC would seamlessly take over, so failover to DR site would generally not be required for this.
The uses case is to monitor and ingest the traffic data from the applications to elastic node and our application is hosted in DC and DR environment, so we have to setup the DC DR environment for Elastic stack also
We need to keep the 30 days of retention period for quick access the reports and dashboards and remaining should be present in warm node without deleting the data.
5gb per day
I’ll take just one of the unanswered questions which is easy. Yes. If you had a big enough server you could have hot, warm, cold, and frozen tiers all running on that one server, via containers, VMs in various guises, or even directly. Try not to forget that elasticsearch is just a java application.
From reading the rest of the thread I am un-convinced this would be sensible in your case, but it is possible.
What is the longest retention period you require? Keeping data indefinately is generally not practical.
This is a very small volume and if we assume the data takes uop the same amount of data on disk it is only 150GB over a 30-day period. A single node can generally handle a lot more than that so I do not see any need for a hot-warm architecture here.
You did not answer my earlier questions around how far apart these DCs are and what the connectivity and latency look like. Could you please provide some clarity on this?
Do you also require redundancy within each DC?
What do you have already? i.e. what part of any of this exists as of today? Are we at the concept/design stage (I am hoping so)?
And “our application is hosted in DC and DR environment” means the app is generating data on both environments, or only one at a time? Is it envisaged that the application and ”elasticsearch cluster” would be failed-over to the secondary site at the same time ?
A diagram would really help me here, but then I like diagrams.