Filebeat filestream input has inconsistent tag output when using "take_over"

Elastic Stack Beats
1

We recently upgraded from beats 7.x and moved to beats 8.x. When we moved from input type of log to filestream, all of our logstash routing broke because the tags are inconsistent in the events. We are using the take_over directive in the input block, and for some events this appears to be overwriting the tags we are sending with the inputs . I have tried putting the tags at the top level of the input, but also within the fields block with fields_under_root and I see to get the same behavior.

Deployed on Ubuntu 22.04
filebeat version 8.19.8 (amd64), libbeat 8.19.8 [1806685c9ad5c94b987c6e7b43eb9b63d3926dec built 2025-11-26 21:54:43 +0000 UTC] (FIPS-distribution: false)

Here is my config block:

type: filestream
id: app-utilization
take_over: true
enabled: true
paths:
  - /var/log/app/utilization*.log
close.reader.after_interval: 5m
tags: ["apputilization"]
fields:
 # tags: ["apputilization"]  # have also tried here with the same result
 appVersion: "1.5.4"
fields_under_root: true

The results I get back from the same host, same file, and even the same batch is that some will have the apputilization tag in them, and some will not. They all have the take_over tag in them though.

2

Hi @mike.guthrie, could you post your full Filebeat configuration? Just make sure to redact any credential. Please, also post the Logstash configuration you’re using.

Is there any change that the Logstash pipelines are not handling correct events with multiple tags?

I did looked at the input code and did a quick test using the file output and on my tests both tags are in the final event.

My full configuration:

filebeat.inputs:
  - type: filestream
    id: app-utilization
    take_over: true
    enabled: true
    paths:
      - /tmp/flog.log
    close.reader.after_interval: 5m
    tags: ["apputilization"]
    fields:
      # tags: ["apputilization"]  # have also tried here with the same result
      appVersion: "1.5.4"
    fields_under_root: true

output.file:
  path: ${path.home}
  filename: output
  rotate_on_startup: false
  codec.json:
    pretty: false

Once the output file is populated, I checked it with cat output-20251205.ndjson |jq '.tags' -c and all events had both tags, here is a snippet of the output:

% cat output-20251205.ndjson |jq '.tags' -c
["take_over","apputilization"]
["take_over","apputilization"]
["take_over","apputilization"]
["take_over","apputilization"]

If the problem is in Filebeat, I suspect it might be related to the publishing workers/output.