Running version 8.12 on Docker Swarm, I want to export all logs from this swarm to a Logstash-type input that runs on Graylog. After a day of work I got data into Graylog but with very, very few fields. I'm guessing that there are members here who have been doing this for years and may be willing to share their config with me.
My current filebeat.yml is:
filebeat.autodiscover:
providers:
- type: docker
templates:
- config:
- type: container
containers.ids:
- "${data.docker.container.id}"
paths:
- "/var/lib/docker/containers/${data.docker.container.id}/*.log"
fields_under_root: true
processors:
- add_host_metadata: ~
- add_docker_metadata: ~
- decode_json_fields:
fields: ["message"]
target: ""
overwrite_keys: true
add_error_key: true
output.logstash:
hosts: ["192.168.194.59:12211"]
In Graylog I see no docker or host data, and the "message" field appears in pure JSON, i.e. it is not parsed as I'd expect.
Please give me a nudge in the right direction, I'm sure my config has lots of room for improvement.
This question is purely about Filebeat; Graylog is my own concern.