Where are Docker metadata?

Elastic Stack Beats
1

I'm using Filebeat in a Docker container to forward logs of every container to a Kafka receiver:

filebeat.prospectors:
- type: log
  paths:
    - '/var/lib/docker/containers/*/*.log'
  json.message_key: log
  json.keys_under_root: true
  multiline.pattern: '^MSG|WNG|ERR'
  multiline.negate: true
  multiline.match: after
  tail_files: true
  processors:
  - add_docker_metadata: ~

output.kafka:
  hosts: ["plogelk01:9092", "plogelk02:9092", "plogelk03:9092"]
  codec.format:
    string: '%{[beat.hostname]} message=%{[log]}'
  topic: 'filebeat_SUBSTENV_logs'
  partition.round_robin:
    reachable_only: false
  required_acks: 1
  compression: gzip
  max_message_bytes: 1000000

Any idea why I'm not able to see any docker.* attribute looking at Kafka messages?
Peraphs, am I skipping them by my custom codec?

2

@mimmus I think you should see them, can you try using output.file instead of the output.kafka, the file output will allow us to know if it's a filebeat or kafka problem.

3

OK, 

codec.format
overrides all fields.
Commenting it, all docker.* attributes are shown.

Thanks

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.