Filebeat central log server and hostname

Elastic Stack Beats
1

I have a central log server receiving all system and audit logs. The system logs all go into the same log file and have the host name in the message. I only have filebeat running on the central log server processing logs for all hosts. The default filebeat and elastic ingest pipeline recognize the host and put it into the related hosts field. But the host name is populated with the central log server host name. It looks like the filebeat default config has a configuration for handling forwarded logs but my logs don't have the required forwarded tag. Can anyone help me understand what needs to be added to my logs and if you know a configuration option in rsyslog to add the required tag. Filebeat is looking for this tag to not populate the host name with the central log server host information.

# /etc/filebeat/filebeat.yml
# ================================= Processors =================================
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded

I could look at some other options by removing the add_host_metadata processor from the main config and then call it in the module processor looking at fields in the parsed log but would like to see if this "built in" config is an option I can use before pursuing an alternate approach.

A follow-on question would be which processors are processed first, the module processors or the general processors found in filebeat.yml? If the module processors are processed first then I could add a tag there and only have the general processor add the host metadata if the log originated on the central log server.

2

I resolved this and posted about it here:
Auditd ingest pipeline with forwarded logs - Elastic Stack / Beats - Discuss the Elastic Stack

For syslogs and authlogs will not need the Digest processor but basically a processor in the individual module runs first. There check to see the original log file path and if not the central log server then add the forwarded tag. Also copied the fields host.hostname to host.name so both are populated correctly

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.