Path.data: is ignored in filebeat.yml ===> Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data)

Elastic Stack Beats
1

Hi

path.data is ignored in filebeat.yml

I am on RHEL9 if that matters.

I try to start 2 different filebeat processes.

the process already running:

# ps -ef | grep filebeat
root      552625  519242  0 14:32 pts/3    00:00:00 less filebeat
root      574795       1  0 15:15 ?        00:00:03 /usr/share/filebeat/bin/filebeat --environment systemd -c /etc/filebeat/filebeat.yml --path.home /usr/share/filebeat --path.config /etc/filebeat --path.data /var/lib/filebeat --path.logs /var/log/filebeat

I wish to start 2nd process with different paths

# filebeat -c /opt/dctm/am/filebeat/filebeat.yml
Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).

config file

# cat /opt/dctm/am/filebeat/filebeat.yml
path.data: "/opt/dctm/filebeat/data"
path.config: "/opt/dctm/am/filebeat"
path.logs: "/opt/dctm/filebeat/logs"

filebeat.config.modules:
  enabled: false
  path: /opt/dctm/am/filebeat/modules.d/*.yml

filebeat.inputs:
- enabled: true
  paths:
    - /opt/dctm/am/filebeat/conf.d/*.yml

output.logstash:
  hosts:
    - "example.com:443"
  username: "example-user"
  password: ""
  ssl.enabled: true
  ssl.insecure: true

logging.level: debug
logging.to_files: true
logging.files:
  path: "/opt/dctm/filebeat/logs"
  name: "filebeat"
  keepfiles: 7
  permissions: 0644

filebeat.config.overwrite: true
filebeat.registry.path: "/opt/dctm/filebeat/registry"

thank you

2

Can you share the contents of the log file created? It should populate several lines on first run even with a data lock file error.

You may also be able to get the log by adding -e to your invocation. Mine logs something like:

bill@bills-MBP metricbeat-8.16.1-darwin-aarch64 % ./metricbeat run -e
{"log.level":"info","@timestamp":"2024-12-17T19:49:50.181-0600","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).configure","file.name":"instance/beat.go","file.line":1058},"message":"Home path: [/Users/bill/Downloads/metricbeat-8.16.1-darwin-aarch64] Config path: [/Users/bill/Downloads/metricbeat-8.16.1-darwin-aarch64] Data path: [/Users/bill/Downloads/metricbeat-8.16.1-darwin-aarch64/data] Logs path: [/Users/bill/Downloads/metricbeat-8.16.1-darwin-aarch64/logs]","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-12-17T19:49:50.181-0600","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).configure","file.name":"instance/beat.go","file.line":1066},"message":"Beat ID: 368def15-35c4-4d4e-834e-7e9879a6b653","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2024-12-17T19:49:51.810-0600","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).launch","file.name":"instance/beat.go","file.line":667},"message":"metricbeat stopped.","service.name":"metricbeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2024-12-17T19:49:51.810-0600","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.handleError","file.name":"instance/beat.go","file.line":1590},"message":"Exiting: /Users/bill/Downloads/metricbeat-8.16.1-darwin-aarch64/data/metricbeat.lock: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data)","service.name":"metricbeat","ecs.version":"1.6.0"}
Exiting: /Users/bill/Downloads/metricbeat-8.16.1-darwin-aarch64/data/metricbeat.lock: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data)