Custom filebeat module

Elastic Stack Beats
1

Filebeat configuration supports multiple inputs, how can be this achived with custom module config.yml ?
I would like to tag single path with the unique value.

type: log
paths:
  - C:/Logs1/*/*.log  -> tag1
  - C:/Logs2/*/*.log  -> tag2

I tried with two input definitions in the same file, but no success.

type: log
paths:
  - C:/Logs1/*/*.log 
tags: [Tag1]

type: log
paths:
  - C:/Logs2/*/*.log 
tags: [Tag2]
2

@mraz1337 I think you were close, but sometime YAML alignment get in the way. Something similar to the following should work.

filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

- type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /var/log/*.log
    #- c:\programdata\elasticsearch\logs\*
    
  # Optional additional fields. These fields can be freely picked
  # to add additional information to the crawled log files for filtering
  fields:
    tags: tag1

- type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /var/other logs/*.log
    #- c:\programdata\elasticsearch\logs\*
    
  # Optional additional fields. These fields can be freely picked
  # to add additional information to the crawled log files for filtering
  fields:
    tags: mytag2
3

I used your example but i get an error in filebeat logs:

ERROR	fileset/factory.go:105	Error creating input: No paths were defined for input accessing config

....

ERROR	instance/beat.go:743	Exiting: No paths were defined for input accessing config
4

Can you add the output of filebeat export config to this thread?

5

Exported config:

filebeat:
  config:
    modules:
      path: d:\Elastic\filebeat-6.4.1-windows-x86_64/modules.d/*.yml
      reload:
        enabled: false
  inputs:
  - enabled: false
    paths: null
    type: log
logging:
  level: debug
output:
  elasticsearch:
    hosts:
    - localhost:9200
path:
  config: d:\Elastic\filebeat-6.4.1-windows-x86_64
  data: d:\Elastic\filebeat-6.4.1-windows-x86_64\data
  home: d:\Elastic\filebeat-6.4.1-windows-x86_64
  logs: d:\Elastic\filebeat-6.4.1-windows-x86_64\logs
setup:
  dashboards:
    enabled: true
  kibana:
    host: localhost:5601
  template:
    settings:
      index:
        number_of_shards: 3

filebeat.inputs: are configured inside module.

6

@pierhugues did you manage to figure something out?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.