Filebeat - duplicated logs when using journald Input with systemd units

Elastic Stack Beats
1

Hello, I've encountered an issue while using the journald input in Filebeat, specifically when specifying individual systemd units. Instead of separating logs based on the systemd unit, Filebeat seems to be processing the entire journal and this leads to duplicated logs with different systemd unit fields.

I tested both of these versions with exact results:

filebeat version 9.3.0 (amd64), libbeat 9.3.0 [0f4fc63162db855e0a1c5f0ec5894a8939e31d80 built 2026-01-29 07:07:30 +0000 UTC]

filebeat version 7.17.29 (amd64), libbeat 7.17.29 [bb9ad7633fb96c506b9fe1723b91d82fc4fb6317 built 2025-06-18 16:33:18 +0000 UTC]

filebeat.yml:

---
filebeat.inputs:
  - type: journald
    id: kernel-dmesg
    include_matches:
      match:
        - _TRANSPORT=kernel
    fields:
      application: dmesg
    fields_under_root: true

  - type: journald
    id: sshd
    include_matches:
      match:
        - _SYSTEMD_UNIT=sshd.service
    fields:
      application: sshd
    fields_under_root: true

  - type: journald
    id: smartd
    include_matches:
      match:
        - _SYSTEMD_UNIT=smartd.service
    fields:
      application: smartd
    fields_under_root: true

  - type: journald
    id: frr
    include_matches:
      match:
        - _SYSTEMD_UNIT=frr.service
    fields:
      application: frr
    fields_under_root: true

  - type: journald
    id: rsyslog
    include_matches:
      match:
        - _SYSTEMD_UNIT=rsyslog.service
    fields:
      application: rsyslog
    fields_under_root: true

queue.disk:
  max_size: 3GB

processors:
  - add_fields:
      target: ''
      fields:
        source: 'REDACTED'

output.logstash:
  hosts:
    - 'REDACTED:5044'

logging.metrics.enabled: false

In both versions, logs that should be specific to individual systemd units are being duplicated across different service fields (e.g. smartd, frr, etc.). For example, I queried for the term “session closed” (a common linux log message) in Graylog, which is not expected to be generated by processes like frr or smartd. However, when I checked the logs, I found identical logs from each service configured in Filebeat (as seen in the attached screenshot).

image
image734×399 12.7 KB

Do you have any clue whether that might be a bug or some misconfiguration on my side? I expected Filebeat to correctly separate logs based on the defined systemd units (sshd, smartd, frr, etc.) and not duplicate logs across them, because even though path in which these logs can be found is identical (/var/log/messages in my case), tailing manually any of these services (using e.g. journalctl -u sshd.service -f does not return logs from other units.

2

Worked with the user on the Elastic community slack. Outputting filebeat to a file showed no dupes. This appears to be a downstream issue not related to the stack, as they are using Graylog and OpenSearch, and the breakdown may be happening somewhere there.