Multiple Filebeat logs path configuration stops the filebeat

[PRASHANT_MEHTA]

Hello All,

1)After I'm configuring multiple filebeat path in filebeat.yml,filebeat gets stopped with below error:

Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).

---

filebeat.inputs:

• type: log
  enabled: true
  paths:

  • /l/logs/CIS/**/*.log

  • /l/pool/CIS/requests/*/log/.*log

---

For single path configured such issue is not faced.what might be the issue?

2. Is it also possible I can define env variable for path,something like below,what would be right approach?

paths:
-${KPDM_SHARED}/MIS/requests/**/log/*.log

---

Filebeat.yml

filebeat.inputs:

• type: log
  enabled: true
  paths:
  • /l/logs/MIS/**/.log
  • /l/pool/CIS/requests/log/*.log
    ignore_older: 1h
    include_lines:
  • ^[1]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3}Z ?(.)
    multiline.type: pattern
    multiline.pattern: ^[2]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3}Z
    multiline.negate: true
    multiline.match: after
    scan_frequency: 30s
    harvester_limit: 100
    close_inactive: 30m
    close_removed: true
    clean_removed: true
    filebeat.config.modules:
    path: ${path.config}/modules.d/.yml
    reload.enabled: false
    setup.ilm.enabled: true
    setup.ilm.check_exists: true
    setup.ilm.rollover_alias: cis-log
    setup.ilm.pattern: '{now/d}-000001'
    setup.ilm.overwrite: false
    setup.kibana:
    host: http://ctd06appl0012.dev.m01.group.g:5600
    output.Elasticsearch:
    hosts:
  • http://ctd06appl0013.dev.m01.group.g:9400
    processors:
• add_host_metadata: null
• drop_fields:
  when:
  equals:
  agent.type: filebeat
  fields:
  • agent.hostname
  • agent.id
  • agent.type
  • agent.ephemeral_id
  • agent.version
  • log.offset
  • log.flags
  • input.type
  • ecs.version
  • host.os
  • host.id
  • host.mac
  • host.architecture
    monitoring.enabled: true
    monitoring.Elasticsearch: null

---

start filebeat:

#!/bin/ksh
echo "Starting filebeat..."
./filebeat &
_pid=$!
echo "$_pid" > filebeat.pid
echo "Pid $_pid stored in filebeat.pid"

---

1. 0-9 ↩︎

2. 0-9 ↩︎

[mtojek (Marcin Tojek)]

Hi! Could you please reformat your post using code tags? It's hard to read it or spot errors now.

[PRASHANT_MEHTA]

@mtojek Thanx for looking into it!

The issue is solved now.

---

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /l/logs/MIS/**/*.log
    - ${KPTI_SHARED}/MIS/requests/*/log/*.log

Still I'm Not sure why below error keeps on coming in terminal and filebeat stops:

Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data)

[zx8086]

It looks to me that both paths intersect.. so you are not harvesting files from two purely separate locations

/MIS/**/*.log
/MIS/requests/