Multiple Filebeat logs path configuration stops the filebeat

Elastic Stack Beats
1

Hello All,

1)After I'm configuring multiple filebeat path in filebeat.yml,filebeat gets stopped with below error:

Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data).

filebeat.inputs:

  • type: log
    enabled: true
    paths:

    • /l/logs/CIS/**/*.log

    • /l/pool/CIS/requests/*/log/.*log

For single path configured such issue is not faced.what might be the issue?

  1. Is it also possible I can define env variable for path,something like below,what would be right approach?

paths:
-${KPDM_SHARED}/MIS/requests/**/log/*.log

Filebeat.yml

filebeat.inputs:

  • type: log
    enabled: true
    paths:
    • /l/logs/MIS/**/.log
    • /l/pool/CIS/requests/log/*.log
      ignore_older: 1h
      include_lines:
    • [1]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3}Z ?(.)
      multiline.type: pattern
      multiline.pattern: [2]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}.[0-9]{3}Z
      multiline.negate: true
      multiline.match: after
      scan_frequency: 30s
      harvester_limit: 100
      close_inactive: 30m
      close_removed: true
      clean_removed: true
      filebeat.config.modules:
      path: ${path.config}/modules.d/      .yml
      reload.enabled: false
      setup.ilm.enabled: true
      setup.ilm.check_exists: true
      setup.ilm.rollover_alias: cis-log
      setup.ilm.pattern: '{now/d}-000001'
      setup.ilm.overwrite: false
      setup.kibana:
      host: http://ctd06appl0012.dev.m01.group.g:5600
      output.Elasticsearch:
      hosts:
    • http://ctd06appl0013.dev.m01.group.g:9400
      processors:
  • add_host_metadata: null
  • drop_fields:
    when:
    equals:
    agent.type: filebeat
    fields:
    • agent.hostname
    • agent.id
    • agent.type
    • agent.ephemeral_id
    • agent.version
    • log.offset
    • log.flags
    • input.type
    • ecs.version
    • host.os
    • host.id
    • host.mac
    • host.architecture
      monitoring.enabled: true
      monitoring.Elasticsearch: null

start filebeat:

#!/bin/ksh
echo "Starting filebeat..."
./filebeat &
_pid=$!
echo "$_pid" > filebeat.pid
echo "Pid $_pid stored in filebeat.pid"

  1. 0-9 ↩︎

  2. 0-9 ↩︎

2

Hi! Could you please reformat your post using code tags? It's hard to read it or spot errors now.

3

@mtojek Thanx for looking into it!

The issue is solved now.

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /l/logs/MIS/**/*.log
    - ${KPTI_SHARED}/MIS/requests/*/log/*.log

Still I'm Not sure why below error keeps on coming in terminal and filebeat stops:

Exiting: data path already locked by another beat. Please make sure that multiple beats are not sharing the same data path (path.data)

4

It looks to me that both paths intersect.. so you are not harvesting files from two purely separate locations


/MIS/**/*.log
/MIS/requests/
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.