Docker module on metricbeat

Elastic Stack Beats
1

The docker module on metric beat shows error when I try to put it in the conf file.
Please let me know if i m wrong with the configuration for the docker module,

This is the error I m getting when I try to run with the docker module:

2018/04/13 05:36:53.954728 beat.go:346: CRIT Exiting: 7 errors: host parsing failed for docker-container: error parsing URL: empty host; host parsing failed for docker-cpu: error parsing URL: empty host; host parsing failed for docker-diskio: error parsing URL: empty host; host parsing failed for docker-healthcheck: error parsing URL: empty host; host parsing failed for docker-info: error parsing URL: empty host; host parsing failed for docker-memory: error parsing URL: empty host; host parsing failed for docker-network: error parsing URL: empty host
Exiting: 7 errors: host parsing failed for docker-container: error parsing URL: empty host; host parsing failed for docker-cpu: error parsing URL: empty host; host parsing failed for docker-diskio: error parsing URL: empty host; host parsing failed for docker-healthcheck: error parsing URL: empty host; host parsing failed for docker-info: error parsing URL: empty host; host parsing failed for docker-memory: error parsing URL: empty host; host parsing failed for docker-network: error parsing URL: empty host

This is the module part which I added on the conf file:

- module: docker
  metricsets: ["container", "cpu", "diskio", "healthcheck", "info", "memory", "network"]
  hosts: ["nceapttls13:///var/run/docker.sock"]
  enabled: true
  period: 10s

  # To connect to Docker over TLS you must specify a client and CA certificate.
  ssl:
    certificate_authority: "/etc/pki/root/ca.pem"
    certificate:           "/etc/pki/client/cert.pem"
    key:                   "/etc/pki/client/cert.key"

Please let me know where m I going wrong.

2

Try changing the URL to hosts: ["unix:///var/run/docker.sock"] (which is the default, IIRC).

3

Still did not work
Is it that we have to activate the module or something or just by putting the module in the yml file is enough.

4

Was there an error when you ran it? Did config test pass (metricbeat test config -e -d "*")? Did the module test pass (metricbeat test modules docker -e -d "*")?

There are two ways to specify the modules that will run -- put it in metricbeat.yml or put it into the modules.d directory. See Configuring Metricbeat » Specify which modules to run.

5

my metricbeat is running on docker as a container
also i m using version 5.6.8
could you give the commands to test whether metric beat is running or not on docker

6

2018/04/17 13:05:45.973230 metrics.go:39: INFO Non-zero metrics in the last 30s: fetches.docker-cpu.events=3 fetches.docker-cpu.failures=3 fetches.docker-diskio.events=3 fetches.docker-diskio.failures=3 fetches.docker-healthcheck.events=3 fetches.docker-healthcheck.failures=3 fetches.docker-info.events=3 fetches.docker-info.failures=3 fetches.docker-memory.events=3 fetches.docker-memory.failures=3 fetches.docker-network.events=3 fetches.docker-network.failures=3 fetches.system-cpu.events=3 fetches.system-cpu.success=3 fetches.system-filesystem.events=105 fetches.system-filesystem.success=3 fetches.system-fsstat.events=3 fetches.system-fsstat.success=3 fetches.system-load.events=3 fetches.system-load.success=3 fetches.system-memory.events=3 fetches.system-memory.success=3 fetches.system-network.events=6 fetches.system-network.success=3 fetches.system-process.events=3 fetches.system-process.success=3 fetches.system-socket.success=60 libbeat.es.call_count.PublishEvents=3 libbeat.es.publish.read_bytes=1447 libbeat.es.publish.write_bytes=69736 libbeat.es.published_and_acked_events=144 libbeat.publisher.messages_in_worker_queues=144 libbeat.publisher.published_events=144

This is the logs which i get This clearly shows that everything with respect to docker is failing

7

You should be able to execute the test command with:

docker exec -it <metricbeat_container_id_or_name> /usr/share/metricbeat/metricbeat test modules docker -e -d "*"

8

What does the "*" refer to as in will it be as it is or we need to put the host address in it

10

-d "*" causes it to log all debug.

11

it is not happening for whatever reason I am not able to understand

12

Should I post my yml file here
In which i tried including the docker module

13

Nothing at all? It should have at least gave some debug output or an error.

As an alternative way to collect information about why it's failing to communicate to Docker, you can enable debug logging in your metricbeat.yml config file.

logging.level: debug
logging.selectors: ['*']

Another thing I noticed is that you are using a unix socket AND you have uncommented the ssl options. I think you can leave the ssl options commented out when using a unix socket. Similar to this example.

14

nothing it is showing

15

Even after applying this it shows nothing boss

16

If you are running Metricbeat inside of a container then did you mount /var/lib/docker.sock inside of the container so that Metricbeat can access it?

If so, then does the Metricbeat process have the appropriate permissions to read/write to the socket?

I did a quick test with docker-compose to see if everything was working OK with the docker module. Here's the config I used in case it helps.

# Dockerfile
FROM docker.elastic.co/beats/metricbeat:6.2.3
COPY metricbeat.yml /usr/share/metricbeat/metricbeat.yml
USER root
RUN chown root /usr/share/metricbeat/metricbeat.yml

# docker-compose.yml
---
version: '3'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.2.3
    environment: ['http.host=0.0.0.0', 'transport.host=127.0.0.1']
    ports: ['127.0.0.1:9200:9200']

  kibana:
    image: docker.elastic.co/kibana/kibana:6.2.3
    container_name: kibana
    ports: ['127.0.0.1:5601:5601']
    depends_on: ['elasticsearch']

  metricbeat:
    build: .
    volumes:
    - /var/run/docker.sock:/var/run/docker.sock
    command: -e 
    depends_on: ['elasticsearch']
    # So that metricbeat has permission to access the Docker socket.
    user: root

# metricbeat.yml
metricbeat.modules:
- module: docker
  metricsets: ["container", "cpu", "diskio", "healthcheck", "info", "memory", "network"]
  hosts: ["unix:///var/run/docker.sock"]
  period: 10s

output.elasticsearch:
  hosts: ["elasticsearch:9200"]

$ docker-compose run
# Wait for things to start.

# See if events are present.
$ curl http://localhost:9200/_cat/indices/metricbeat*?v
health status index                       uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   metricbeat-6.2.3-2018.04.17 GGotq8MUT9m88xMMFpX1pg   5   1        398            0      812kb          812kb

# To see some of the raw events.
$ curl http://localhost:9200/metricbeat-*/_search?pretty
17

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.