Filebeat live loading config not work

Elastic Stack Beats
1

Hi, I'm beginner of filebeat.

When I use filebeat live config, filebeat ignores config file.

filebeat version: 6.2.2

my filebeat.yml

filebeat.config.inputs:
  enabled: true
  path: /usr/share/filebeat/inputs.d/*.yml
  reload.enabled: true
  reload.period: 10s

and /usr/share/filebeat/inputs.d/config.yml

    filebeat.prospectors:
    - type: log
      paths:
        - "/logs/*.log"
      json.keys_under_root: true
      fields:
        log_topic: TEST
    output.kafka:
      hosts: ["kafka1:9090","kafka2:9090","kafka3:9090"]
      topic: '%{[fields.log_topic]}'
      keep_alive: 5
      compression: gzip

And Error

2018-10-16T01:51:34.203Z        INFO    instance/beat.go:468    Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs]
2018-10-16T01:51:34.203Z        INFO    instance/beat.go:475    Beat UUID: 5dd7ebb2-a224-43d0-ba86-33c6945b3c19
2018-10-16T01:51:34.203Z        INFO    instance/beat.go:213    Setup Beat: filebeat; Version: 6.2.2
2018-10-16T01:51:34.203Z        INFO    pipeline/module.go:91   No outputs are defined. Please define one under the output section.
2018-10-16T01:51:34.203Z        ERROR   instance/beat.go:667    Exiting: error initializing publisher: No outputs are defined. Please define one under the output section.
Exiting: error initializing publisher: No outputs are defined. Please define one under the output section.

I load confguation with k8s configmap.

When I use config file below, it works well.

filebeat.yml

    filebeat.prospectors:
    - type: log
      paths:
        - "/logs/*.log"
      json.keys_under_root: true
      fields:
        log_topic: TEST
    output.kafka:
      hosts: ["kafka1:9092","kafka:9092"]
      topic: '%{[fields.log_topic]}'
      keep_alive: 5
      compression: gzip

Am I do something wrong?

2

Filebeat does not support full live reconfiguration. Filebeat supports auto-discovery for inputs and modules. The filebeat.config.inputs setting is a special input output discovery mode reading new input-only configurations from filebeat.

e.g.

filebeat.yml:

filebeat.config.inputs:
  enabled: true
  path: /usr/share/filebeat/inputs.d/*.yml
  reload.enabled: true
  reload.period: 10s

output.kafka:
      hosts: ["kafka1:9092","kafka:9092"]
      topic: '%{[fields.log_topic]}'
      keep_alive: 5
      compression: gzip

and /usr/share/filebeat/inputs.d/logs.yml:

- type: log
  paths:
    - "/logs/*.log"
  json.keys_under_root: true
  fields:
    log_topic: TEST

See: https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-configuration-reloading.html#filebeat-configuration-reloading

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.