Collecting docker logs using filebeat

Elastic Stack Beats
1

Hi,

I want to collect logs(with container metadata like container id, image name, host machine) of applications running in containers deployed on more than one host machines using Mesos-Marathon. After some initial investigation, I have decided to use elastic search, filebeat and Kibana.
Which way is good to go:

  1. Running filebeat inside each container, or
  2. Running filebeat per host, or
  3. any other way?

If we select option 2 above, how to add container metadata with logs? If I make one prospector per container with dynamically populated tags/fields as that container's metadata, how to continuously check for any new containers coming up and add corresponding new prospectors?

2

I would definitively go with option 2. For the meta data: You would need a script that checks the container stats to check for new containers or have each new container write its config file itself (trough a script). We are aware that this is not optimal yet and working on ideas on how we could improve this.

3

Thanks @ruflin for your reply.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.