Data path already locked by another beat

Elastic Stack Beats
1

after upgrading from 7.5.1 to 7.6.0 my filebeat breaks with error data path already locked by another beat

i have two instances of filebeat running for a very long time, one reading a specific set of files and sending to logstash, and the other one reading another set of files and sending to elasticsearch.

could not find this breaking change on the docs below.

https://www.elastic.co/guide/en/beats/libbeat/7.6/breaking-changes-7.6.html
and
https://www.elastic.co/guide/en/beats/libbeat/7.6/release-notes-7.6.0.html

2

filebeat.elasticsearch.yml

filebeat.registry.path: registry.elasticsearch

processors:
- add_host_metadata:
    netinfo.enabled: true

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true
  reload.period: 1h

filebeat.modules:
- module: apache
  access:
    var.paths: ["/var/log/apache2/*/access.log"]
  error:
    var.paths: ["/var/log/apache2/*/error.log"]

output.elasticsearch:
  compression_level : 5
  hosts            : elastic-zmb-pool
  loadbalance       : true
  worker            : 2
3

filebeat.logstash.yml

filebeat.registry.path: registry.logstash

processors:
- add_host_metadata:
    netinfo.enabled: true

filebeat.inputs:
- type              : log
  enabled           : true
  exclude_files     : ['.gz$']
  fields_under_root : true
  tags              : ["zimbra"]
  paths:
    - /opt/zimbra/log/access_log.*
    - /opt/zimbra/log/audit.log
    - /opt/zimbra/log/mailbox.log
    - /opt/zimbra/log/nginx.log
    - /opt/zimbra/log/nginx.access.log
    - /opt/zimbra/log/sync.log
    - /var/log/zimbra.log

output.logstash:
  hosts             : elastic-zmb-pool
  loadbalance       : true
4

Hmmm, you're right that this important breaking change is not in the release notes or breaking changes doc for 7.6.0. I'm terribly sorry about that and am working to get it documented ASAP!

As for fixing your setup to account for this change, please edit both your Filebeat configuration files and add a path.data setting. Make sure to set it's values to unique directory locations. Then you should set filebeat.registry.path to start with ${path.data}/.

Shaunak

5

Are you referring here to this path.data?
https://www.elastic.co/guide/en/beats/filebeat/master/configuration-path.html#_data

1 Like
6

That's correct @florinsfetea.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.