Logstash or Beats

Elastic Stack Beats
1

I would like an experienced opinion on whether to use Logstash or Beats to ship logs to Elasticsearch for an enterprise network. What is the advantage of using either and of using both?
I want to be able to ship windows system and security events from over 10 Windows servers and to ship filebeat and packetbeat logs from Zeek and Wazuh. What do I ship to logstash with beats and what do I ship directly to elasticsearch with beats?

2

I've been using both. I like the extra processing I can do in logstash, so I've been using beats to send to logstash, and logstash processes and then stores to elasticsearch. Doing this enables me to use the aggregate filter to add extra context to other logs. For example, I can store the username when a user logs in from a specific IP, then add it to future DNS log events. It's more complicated this way, but makes for nicer reports. For security reasons, I also like firewalling off my elasticsearch ports to only the required hosts. If winlogbeats is sending through logstash, I can do this for a little more peace-of-mind.

If you're looking for a quicker or easier solution, using beats is probably the easiest/fastest to get going.

3

Thanks Ian. That is helpful.
Does sending Windows events to logstash have any impact on indexing and can it impose a delay on an active search e.g. If I was looking for Sysmon activity in real time.

4

I just checked my logs from windows workstations. It looked like if I refresh a 1 minute search, my results are about 10-15 seconds old. You may have better results, I do a lot of crazy things with logstash that eat CPU time:

  • Lookup selected IP addresses in my ip address manager
  • Resolve IP to username (based on most recent logon)
  • Resolve IP to hostname (based on most recent known host at that IP (NOT dns))
  • Geo-IP specific IP address fields
  • Aggregate data in some logs into a single log entry
  • 120ish grok rules
  • Some field translations (turns event codes into text)
  • Key value parsing for some log entries
  • Removing unneeded fields

A lot of those things can be done using beats and elasticsearch, but I started with ELK before elastic had beats and before elastic had support for grok.

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.