I want to collect logs(with container metadata like container id, image name, host machine) of applications running in containers deployed on more than one host machines using Mesos-Marathon. After some initial investigation, I have decided to use elastic search, filebeat and Kibana.
Which way is good to go:
- Running filebeat inside each container, or
- Running filebeat per host, or
- any other way?
If we select option 2 above, how to add container metadata with logs? If I make one prospector per container with dynamically populated tags/fields as that container's metadata, how to continuously check for any new containers coming up and add corresponding new prospectors?