Good evening everyone,
As part of a new project within my company we decided to use ELK stack.
One of the pre-requisites of such project is that all components shall have HA configured.
For ElasticSearch HA comes by default, but for Logstash and Kibana we didn't realize and easy way to make it (yes we can use Linux OS Cluster approach), but project see that approach as no flexible
And alternative is being considered and consists on configure a Mesos environment (3 master, 3 slaves) with marathon running on top. (we've just deployed such environment using latest versions available for mesos and marathon)
Our plan is to deploy 3 instances of ElasticSearch, 1 of Logstash and 1 of Kibana, each of them as a docker containers
At least logstash and kibana servers must be accessible by systems outside the mesos cluster, which as far as my understand means we need to set a mesos-dns system with a configuration in our global dns in order to foward each request to mesos cluster to our mesos-dns server
I've reading multiple information about docker containers over marathon, but to be honest I've not a clear clue of howto configure elasticsearch,logstash and kibana docker containers to run on top of marathon over mesos cluster
If anyone can give any tip or point me to some document that can help me to configure the ELK docker containers to run on top of marathon will be great
As well any info about mesos-dns will be much welcome.
Thanks in advance for your help and support