Docker container logs and filebeat

Elastic Stack Beats
1

Hi,

was looking forward to filebeats multiline support and shipping docker container logs to ELK. The only option seems to be to parse /var/lib/docker/containers for json logs. Json and multiline seems hard to do. Also makes it messy having to use grok a lot in logstash. Suggestions?

/Tess

2

We are looking for ways of reliably sending (by that I mean at least once guarantees, like filebeat and logstash usually do) out of Docker containers, but unfortunately the current design of the Docker logging drivers don't make that easy.

One option is to better support the approach that you are trying by adding JSON decoding support in Filebeat, but that approach still has other issues, for example with potentially losing logs when a container is removed.

An approach that I hear works pretty good today is to use the syslog driver to output to a local syslog, that writes and rotates files, that are shipped by Filebeat. A bit more moving parts that we'd like to have, but should work until we have a better solution.