Hi there
I'm pretty new on docker and elasticsearch and I'm trying to install a supervision for my company.
The goal is to have one server hosting both elasticsearch and kibana and many node that are running beats.
I started with elasticsearch, kibana and metricbeat as one mvp installed on my desktop (ubuntu 18.04).
Elasticsearch and kibana are running well, each of them in their containers (I can curl both of them and access to kibana with my browser).
But, metricbeat (container) cannot connect to Elasticsearch.
here is my docker-compose.xml for elasticsearch & kibana
---
version: '3.6'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:${TAG}
hostname: elasticsearch
container_name: elasticsearch
ports:
- "9200:9200"
- "9300:9300"
networks: ['stack']
volumes:
- 'es_data:/usr/share/elasticsearch/data'
healthcheck:
test: curl -s https://elasticsearch:9200 >/dev/null; if [[ $$? == 52 ]]; then echo 0; else echo 1; fi
interval: 30s
timeout: 10s
retries: 5
kibana:
image: docker.elastic.co/kibana/kibana:${TAG}
hostname: kibana
container_name: kibana
ports: ['5601:5601']
networks: ['stack']
depends_on: ['elasticsearch']
healthcheck:
test: curl -s https://kibana:5601 >/dev/null; if [[ $$? == 52 ]]; then echo 0; else echo 1; fi
interval: 30s
timeout: 10s
retries: 5
networks: {stack: {}}
# use docker volume to persist ES data outside of a container.
volumes:
es_data:
$TAG is set with .env file (TAG=6.4.3).
I run it with a single docker-compose up -d
For metricbeat here is my metricbeat.yml configuration file
output.elasticsearch:
hosts: ["elasticsearch:9200"]
protocol: "http"
setup.kibana:
hosts: "kibana:5601"
protocol: "http"
metricbeat.modules:
- module: system
metricsets:
- cpu # CPU usage
- load # CPU load averages
- memory # Memory usage
- network # Network IO
- process # Per process metrics
- process_summary # Process summary
- uptime # System Uptime
#- core # Per CPU core usage
#- diskio # Disk IO
#- filesystem # File system usage for each mountpoint
#- fsstat # File system summary metrics
#- raid # Raid
#- socket # Sockets and connection info (linux only)
enabled: true
period: 10s
processes: ['.*']
# Configure the metric types that are included by these metricsets.
cpu.metrics: ["percentages"] # The other available options are normalized_percentages and ticks.
core.metrics: ["percentages"] # The other available option is ticks.
and the docker-compose.yml
---
version: '3.6'
services:
metricbeat:
image: docker.elastic.co/beats/metricbeat:${TAG}
environment:
- cluster.name=docker-cluster
container_name: metricbeat
#networks: ['stack']
user: root
volumes:
- './metricbeat/metricbeat.yml:/usr/share/metricbeat/metricbeat.yml'
- /proc:/hostfs/proc:ro
- /sys/fs/cgroup:/hostfs/sys/fs/cgroup:ro
- /:/hostfs:ro
healthcheck:
test: metricbeat test config
interval: 30s
timeout: 15s
retries: 5
Once every "module" is running I got this output with a docker ps -a
79a57ccca6fb docker.elastic.co/beats/metricbeat:6.4.3 "/usr/local/bin/dock…" About a minute ago Up About a minute (healthy)
762ca9b0502e docker.elastic.co/kibana/kibana:6.4.3 "/usr/local/bin/kiba…" About an hour ago Up About an hour (healthy) 0.0.0.0:5601->5601/tcp
c0f38f5212da docker.elastic.co/elasticsearch/elasticsearch:6.4.3 "/usr/local/bin/dock…" About an hour ago Up About an hour (healthy) 0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp
Everything seems running
when I attach the container "docker exec -it metricbeat bash" and type "metricbeat export config"
I have this output
metricbeat:
modules:
- core:
metrics:
- percentages
cpu:
metrics:
- percentages
enabled: true
metricsets:
- cpu
- load
- memory
- network
- process
- process_summary
- uptime
module: system
period: 10s
processes:
- .*
output:
elasticsearch:
hosts:
- elasticsearch:9200
protocol: http
path:
config: /usr/share/metricbeat
data: /usr/share/metricbeat/data
home: /usr/share/metricbeat
logs: /usr/share/metricbeat/logs
setup:
kibana:
hosts: kibana:5601
protocol: http
but, still from this container, when I'm trying to connect to the elasticsearch container it fails
metricbeat test output -e -d "*" will give
2018-11-20T11:21:06.506Z INFO instance/beat.go:565 Home path: [/usr/share/metricbeat] Config path: [/usr/share/metricbeat] Data path: [/usr/share/metricbeat/data] Logs path: [/usr/share/metricbeat/logs]
2018-11-20T11:21:06.506Z DEBUG [beat] instance/beat.go:592 Beat metadata path: /usr/share/metricbeat/data/meta.json
2018-11-20T11:21:06.506Z INFO instance/beat.go:572 Beat UUID: d34f8a92-ebfe-4746-9ef4-63f23d856fa0
2018-11-20T11:21:06.506Z INFO elasticsearch/client.go:163 Elasticsearch url: http://elasticsearch:9200
elasticsearch: http://elasticsearch:9200...
parse url... OK
connection...
parse host... OK
dns lookup... OK
addresses: 127.0.0.1
dial up... ERROR dial tcp 127.0.0.1:9200: connect: connection refused
same for
curl http://127.0.0.1:9200/_cat/health
curl: (7) Failed connect to 127.0.0.1:9200; Connection refused
The thing I fail to understand is that when I leave the container curl is accepted
curl http://127.0.0.1:9200/_cat/health
1542713001 11:23:21 docker-cluster yellow 1 1 11 11 0 0 10 0 - 52.4%
Any help will be greatly appreciated