How to implement Logstash filters to handle different logfile formats/types

Elastic Stack Logstash
1

I am ingesting a number of logs including /var/log/syslog, /var/log/messages, but also some proprietary logs in Common Event Format (CEF) and still other proprietary logs in various other formats. The entries of these logs are shot to Logstash from Filebeat running on various nodes, e.g.:

filebeat.inputs:
  - type: log
    enabled: true
    paths:
      - /var/log/messages
      - /var/log/syslog
      - /var/log/acme/audit.log
      - /var/log/acme/debug.log
      - /var/log/acme/console.log

I have written dissect filters for these more proprietary logs. By itself, each filter works, remapping the message field, exploding key-value pairs, etc. to just what I want.

What I don't know how to do is configure Logstash with all of these dissect filters active at once, either by running one log on the right dissect code log by filesystem path (or origin) or by noting failure of one dissect clause and trying a different one instead.

Can this be done in Logstash in the way that I imply? If not, is there configuration in Filebeat that can differentiate the log entries before they reach Logstash in such a way as for Logstash to run different filters for different log entries?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.