My App servers are deployed across 2 AWS regions, and I want to analyze their logs.
I am planning architecture with high availability (1 region failure). So, planning to have:
- Filbeat on each app server,
- 2 Redis (1 in each region),
- 2 Logstash Indexers (1 in each region),
- Elasticsearch cluster spanning across 2 regions,
- 2 Kibana (1 in each region).
Picture of architecture :
I have below questions:
- Can Filebeat setup on app servers in one region communicate with Redis node on another region? Is this setup same as setting up redis-ouput plugin? (providing hosts parameter with IP address of 2 Redis servers) or anything tricky about it.
- Can Logstash Indexer ingest events from Redis server on another region? (is this same as providing 2 Redis input plugins for 2 Redis servers)
- Can Logstash send events to Elasticsearch Data node on another region?
- Can ES cluster span across 2 regions like the picture above? If I set min.master.nodes=3 this can handle 1 region failures, without causing split-brain problem? Will this continue to serve well once the region comes back? (This is where I have concerns about).
Any help is appreciated.