Connection Refused for Metricbeat <-> ES on Docker

Elastic Stack Beats
1

I am trying to use metricbeat and ES in a docker environment. Both are running, but not communicating. Here are the docker logs for the docker containing the metricbeat image:

2018/01/15 19:45:11.260059 beat.go:426: INFO Home path: [/usr/share/metricbeat] Config path: [/usr/share/metricbeat] Data path: [/usr/share/metricbeat/data] Logs path: [/usr/share/metricbeat/logs]
2018/01/15 19:45:11.260285 metrics.go:23: INFO Metrics logging every 30s
2018/01/15 19:45:11.269080 beat.go:433: INFO Beat UUID: 1b68d923-3c70-46d1-96af-4c2e6fb598bb
2018/01/15 19:45:11.273874 beat.go:192: INFO Setup Beat: metricbeat; Version: 6.0.0
2018/01/15 19:45:11.275075 client.go:123: INFO Elasticsearch url: https://drv_elastic:9200
2018/01/15 19:45:11.290819 module.go:80: INFO Beat name: drv_metricbeat
2018/01/15 19:45:11.295296 cfgwarn.go:11: WARN BETA: The docker container metricset is beta
2018/01/15 19:45:11.296141 cfgwarn.go:11: WARN BETA: The docker cpu metricset is beta
2018/01/15 19:45:11.296730 cfgwarn.go:11: WARN BETA: The docker diskio metricset is beta
2018/01/15 19:45:11.297619 cfgwarn.go:11: WARN BETA: The docker healthcheck metricset is beta
2018/01/15 19:45:11.299528 cfgwarn.go:11: WARN BETA: The docker info metricset is beta
2018/01/15 19:45:11.299851 cfgwarn.go:11: WARN BETA: The docker memory metricset is beta
2018/01/15 19:45:11.303504 cfgwarn.go:11: WARN BETA: The docker network metricset is beta
2018/01/15 19:45:11.306910 beat.go:260: INFO metricbeat start running.
2018/01/15 19:45:13.536331 output.go:74: ERR Failed to connect: Get https://drv_elastic:9200: dial tcp 172.18.0.7:9200: getsockopt: connection refused
2018/01/15 19:45:15.538257 output.go:74: ERR Failed to connect: Get https://drv_elastic:9200: dial tcp 172.18.0.7:9200: getsockopt: connection refused
2018/01/15 19:45:19.539823 output.go:74: ERR Failed to connect: Get https://drv_elastic:9200: dial tcp 172.18.0.7:9200: getsockopt: connection refused
2018/01/15 19:45:27.540879 output.go:74: ERR Failed to connect: Get https://drv_elastic:9200: dial tcp 172.18.0.7:9200: getsockopt: connection refused
2018/01/15 19:45:41.269567 metrics.go:39: INFO Non-zero metrics in the last 30s: beat.memstats.gc_next=20196432 beat.memstats.memory_alloc=15517632 beat.memstats.memory_total=50474144 libbeat.config.module.running=0 libbeat.output.type=elasticsearch libbeat.pipeline.clients=2 libbeat.pipeline.events.active=612 libbeat.pipeline.events.published=612 libbeat.pipeline.events.retry=93 libbeat.pipeline.events.total=612 metricbeat.docker.container.events=101 metricbeat.docker.container.success=101 metricbeat.docker.cpu.events=70 metricbeat.docker.cpu.success=70 metricbeat.docker.diskio.events=104 metricbeat.docker.diskio.success=104 metricbeat.docker.info.events=3 metricbeat.docker.info.success=3 metricbeat.docker.memory.events=104 metricbeat.docker.memory.success=104 metricbeat.docker.network.events=104 metricbeat.docker.network.success=104 metricbeat.system.cpu.events=3 metricbeat.system.cpu.success=3 metricbeat.system.filesystem.events=105 metricbeat.system.filesystem.success=105 metricbeat.system.memory.events=3 metricbeat.system.memory.success=3 metricbeat.system.network.events=12 metricbeat.system.network.success=12 metricbeat.system.process.events=3 metricbeat.system.process.success=3
2018/01/15 19:45:43.542609 output.go:74: ERR Failed to connect: Get https://drv_elastic:9200: dial tcp 172.18.0.7:9200: getsockopt: connection refused
2018/01/15 19:45:43.542846 logger.go:22: INFO retryer: send wait signal to consumer
2018/01/15 19:45:43.542876 logger.go:22: INFO done

1. I believe the issue is with the bolded line above. Any ideas for fixes?
2. What is the 172.18.0.7 ip address? I did not explicitly set this.

metricbeat.yml:

metricbeat.modules:

  • module: system
    metricsets:
    • cpu
    • filesystem
    • memory
    • network
    • process
      enabled: true
      period: 10s
      processes: ['.*']
      cpu.metrics: ["percentages"]
  • module: docker
    metricsets:
    • container
    • cpu
    • diskio
    • healthcheck
    • info
    • memory
    • network
      hosts: ["unix:///var/run/docker.sock"]
      enabled: true
      period: 10s
      processors:
    • drop_fields:
      fields:
      • "system.network.in"
      • "system.network.out"
        output.elasticsearch:
        hosts: ["https://drv_elastic:9200"]

partial docker-compose.yml:

DRV ELASTIC
drv_elastic: 
    restart: on-failure:1 
    hostname: drv_elastic 
    build: ./drv_forest/drv_elastic 
    environment: 
      - cluster.name=drv-elastic-cluster 
      - bootstrap.memory_lock=true 
      - "ES_JAVA_OPTS=-Xms1g -Xmx1g -Des.enforce.bootstrap.checks=true" 
    ulimits: 
      memlock: 
        soft: -2 
        hard: -2 
    mem_limit: 2g 
    volumes: 
      - drvstore_elastic_es:/usr/share/elasticsearch/data 
    ports: 
      - 9200:9200
DRV KIBANA
drv_kibana: 
    restart: on-failure:1 
    hostname: drv_kibana 
    build: ./drv_forest/drv_kibana 
    ports: 
      - 5601:5601 
    environment: 
      SERVER_NAME: kibana.example.org 
      ELASTICSEARCH_URL: http://drv_elastic:9200 
      ELASTICSEARCH_USERNAME: kibana 
      ELASTICSEARCH_PASSWORD: login
DRV METRICBEAT
drv_metricbeat: 
    restart: on-failure:1 
    hostname: drv_metricbeat 
    build: ./drv_forest/drv_metricbeat 
    volumes: 
      - /var/run/docker.sock:/var/run/docker.sock 
    depends_on: 
      - drv_mine_mongo 
      - drv_elastic
2

Hi @laubnyc,

Probably Elasticsearch is binding to localhost, you may need to update set network.host to 0.0.0.0. Check https://www.elastic.co/guide/en/elasticsearch/reference/6.1/modules-network.html#common-network-settings for more details.

3

This topic was automatically closed after 21 days. New replies are no longer allowed.