I do have a single instance of ElasticSearch 5.x up and running and did create in 4 weeks about 30 Million documents. The performance is great and I was able to evaluate everything that is needed. But it is difficult to tell how many documents will be created after the global rollout.
I now need to setup a starting infrastructure to rollout the ElasticSearch globally in the whole company.
We do have 9 offices in 4 countries and are all connected via VPN connections.
We want a central monitoring (dashboards) of the major IT systems on all sites. The question is now how to setup a good infrastructure that we can scale out if necessary. I can't find any information about a good design.
Central cluster and all data nodes in one data center ? Or can I split this over different sites ?
Redis installed on each each data source and in addition in front of the ElasticSearch cluster ? I need to cover the case that a site gets disconnected. Of course I don't want to loose the information that gets produced during an outage.
I'm a little bit lost and don't really know where to start.
Any advice or recommendation ?