Optimal way for monitoring host and docker containers metrics

Elastic Stack Beats
1

Hi

I am trying my hands on MetricBeat and have been able to send the host system metrics to an Elastic Search cluster. But, I am facing issues with sending docker containers metrics by metric beat. Also, I am not really sure as to how should I run metric beat (as a separate docker container or as a standalone application) for my use case.

My use case is: I have a host machine (on cloud), and I have multiple docker containers (of applications) running on a single host. Now, I want to send the docker metrics as well as the host metrics to an elastic search cluster (set up across different machines) . Now, I have a few questions:

  1. Is it possible to configure different docker containers in metricbeat.yml file?
  2. Is it better if I run metric beat as a docker container as compared to a application on the host machine? If yes, will I have to link all the different application containers and the host so that it is able to send the docker container metrics and the system metrics? (I am facing issues with running metric beat like this)

Thanks!

2

Hello Bro, see it: https://bitbucket.org/UrielRicardo/docker-elk
In this repo i make it

3

As mentioned in https://github.com/elastic/beats/issues/2718#issuecomment-252176190 you need the snapshot builds for the docker module. https://beats-nightlies.s3.amazonaws.com/index.html?prefix=metricbeat/

  1. What do you mean by that? The docker module monitors all containers of one docker host / machine. So the answer is I think yes.
  2. Both is possible and has its pros and cons. It also depends if you have access to your docker host or not. Running it in a container will probably make it easier to update to you have to mount the correct socket etc. For the system metrics, you refer to the ones of the host machine?

Metricbeat also supports cgroup. Here is a blog post with much more details on it: https://www.elastic.co/blog/monitoring_container_resource_usage_with_metricbeat

@Uriel_Ricardo It seems like the container you have in your repo does not include metricbeat.

4

Ops, wrong repo jajajaja sorry!

5

@ruflin, just a few more doubts:

  1. Is there any documentation as to how can I configure the docker module and what all metricsets can/should I use similar to one that is provided here (https://www.elastic.co/guide/en/beats/metricbeat/master/metricbeat-modules.html)?

  2. So, I have been trying to run metricbeat in a docker container and I am facing problems with this. Is there any official docker image for the metricbeat? Secondly, I am not sure how the metric sets for different modules (docker & system) have to be passed while running the metric beat docker container. I did refer to the page "Running metricbeat in a container", but I still face some issues while running it. I tried using the following https://hub.docker.com/r/fairus/metricbeat/ but have the following errors while running it.

WARN Can not index event (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse [system.process.cpu.start_time]","caused_by":{"type":"illegal_argument_exception","reason":"Invalid format: "05:28" is malformed at ":28""}

I know, this is a third party Dockerfile and it's difficult to comment on it without getting into it's details. Can you maybe point me in the right direction for using it with a docker container?

6

The docker module was just added to metricbeat a few days ago and is still under heavy development. It is on our todo list to add better documentation for it. Currently it is best to follow the docs from dockbeat (found here: https://github.com/Ingensi/dockbeat) as this is where the module is coming from :slight_smile: Please be aware that you are running a snapshot build and we are more then happy to get feedback on it, but there will be breaking changes.

Currently there are not official docker images for beats. As the image is 2 months old that you provided I assume it is not up-to-date and does not have the docker module inside. I strongly recommend do build your own image at the moment or to make it simpler for your, run it directly on the host itself.

7

@ruflin, alright. I will use it as a standalone application on the host for the time being. But just a quick clarification, any docker file for metric beats which are using metric beat 5.0.0-beta1 version won't have support for the docker module, right?

But, does monitoring a separate container from a metric beat container also requires the docker module? Nothing of this sort has been mentioned here: https://www.elastic.co/guide/en/beats/metricbeat/master/running-in-container.html

Also, on the above link, in the command where it mentions the step for monitoring a docker container from a separate metric beats container,

sudo docker run
--link some-mysql:mysql \
-e MYSQL_PASSWORD=secret \
my/metricbeat:latest

Here, "some-mysql" "mysql" is the name and tag value of the docker container we want to monitor?

8

For the metricbeat version:

  • Docker module will probably be in the 5.1 releases
  • cgroups for container monitoring is arleady in the 5.0 releases but is marked experimental

What do you mean by monitoring an other container? If you install metricbeat in each container then you can monitor all the containers. Or you can use cgroups.

The docs for linking containers can be found here: https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/#/communication-across-links

9

This topic was automatically closed after 21 days. New replies are no longer allowed.