Filebeat + Logstash + ElasticSearch

[ryveradam (Adam)]

Greetings fellow Logstashers. I am fairly new to Logstash. I am trying to implement Persisted Queues but I am having trouble understanding some stuff.

1. If my understand is accurate, I would have Logstash running on each instance with the "Beats" plugin installed and configured to push the data to the Logstash Persisted Queues instance. Is this correct?

2. Does anyone know if an example configuration for a host that is sending it's logs to the Logstash Persisted Queue instance?

3. Do I still have to run Logtash on the hosts I want to run Filebeat on to push the logs the Logstash Persisted Queue instance or can I just run Filebeat?

4. Does Lumberjack still have a place here? The Logstash Persisted Queue instance is running the latest version of Logstash with Lumberjack configured to listen on 5044.

Thank you!

[andrewkroh]

The only thing you need to do special from a default setup is to enable the persistent queue feature in your Logstash config.

You could follow the Filebeat getting started guide. The basic architecture will be a Filebeat instance running on each host producing logs. One (or more) central instance(s) of Logstash that receives data from Filebeat using the beats input (not lumberjack). And an Elasticsearch cluster to receive the data from Logstash.