What is the best setup for Beats and ELK stack

This is the setup that I'm currently trying to achieve few months ago with Logstash Forwarder:
LF --> Logstash --> Redis -> Logstash Indexer --> Elasticsearch --> Kibana
However, as Logstash Forwarder will be depreciated soon , I will replace it with Beats tools(Filebeat, packetbeat).
I can see Beats has an output configuration directly to ElasticSearch.
On each monitored node, I will install filebeat to ship certain logs and packetbeat to capture network packets info .

Should I still use the same setup above or connect Beats directly ES ?

What would be the recommended setup?


Cong Nguyen

For packetbeat and topbeat, it is certainly fine to forward events directly to ES as they have boilerplate format and template. However, with filebeat, I would recommend using logstash to "massage the data".

1 Like

The recommended setup is as @vtst2412 described it:

  • Topbeat and Packetbeat directly to Elasticsearch
  • Filebeat through logstash if you need your log files analysed
1 Like

thanks, guys.