We have deployed multiple Logstash instances to separate geolocations and there are multiple Beats instances connected to each Logstash instance in their own geolocation. In a way, it is a distributed setup to different sites, so each site has their own set of Beats and one Logstash instance. All the Logstash instances are connected to a central Elasticsearch cluster.
None of the Beats are allowed to talk directly to the Elasticsearch cluster, only Logstash instances are connected to the Elasticsearch cluster directly.
With this in mind, it would be really handy if there would be a way to enable Beats monitoring and centralized configuration through Logstash, so we wouldn't need to enable port forward on each of the Logstash machines, but we could just add a new input and output in Logstash configurations in order to enable Beats management through Logstash.
Is this something you have been thinking about, or what are your thoughts about my suggestion?
Hello @admlko, I think its something that could be interesting to have especially since a lot of deployment includes air gap system and different zone.
Right now we do not have short terms plan to support proxying configuration from Logstash to a beats instance.
When you are using Central management the beats effectively talks to Kibana and not Elasticsearch directly, so as a deployment strategy I would probably create a ES/Kibana setup responsible for central management and to contains the monitoring data.
I invite you to create an enhancement request on the beats repo to track the work on that feature. https://github.com/elastic/beats/
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.