Filestream take_over mode seems to be ignored

Elastic Stack Beats
1

I'm running filebeat 8.8.2 on centos 7

I'm in the process of switching from log inputs to filestream inputs on pre-existing servers so I was trying to use the take_over mode in order to not re-process the logs.

I'm not sure if it's relevant but I'm using directory of yml files to represent the inputs rather than them being defined in a single file.

The original input was

- type: log
  enabled: true
  paths:
    - /var/log/*.log
    - /var/log/messages
    - /var/log/secure
    - /var/log/maillog
    - /var/log/up2date
    - /var/log/dmesg
    - /var/log/cron
  fields:
    log_type: system
  
  close_renamed: true
  fields_under_root: true
  exclude_files:
    - /var/log/haproxy-err.log
    - /var/log/haproxy-info.log
    - /var/log/haproxy-0.log
    - /var/log/ldap.log*

and the filestream version

- type: filestream
  enabled: true
  id: system
  take_over: true
  paths:
    - /var/log/*.log
    - /var/log/messages
    - /var/log/secure
    - /var/log/maillog
    - /var/log/up2date
    - /var/log/dmesg
    - /var/log/cron
  fields:
    log_type: system
  
  close.on_state_change.renamed: true
  fields_under_root: true
  prospector.scanner.exclude_files:
    - /var/log/haproxy-err.log
    - /var/log/haproxy-info.log
    - /var/log/haproxy-0.log
    - /var/log/ldap.log*

When I restart filebeat after making the configuration change it re-reads the files from the start as if take_over is not set to true

I've run filebeat with logs being output to stdout in debug mode and based on my cursory read of the code I've not found any evidence of the log lines I should see if it is attempting to take over. Also of note the registry directory doesn't have a backup in it which also suggests it's not trying to take over

2

The issue appears to be with the takeover code occurring before it reads in the inputs from separate files. When I move the input into the primary filebeat.yml file it does successfully take over.

Edit: It looks to be patched in 8.10.1

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.