Elasticsearch doesn't create Metricbeat index

Elastic Stack Beats
1

So, i am making a POC Swarm with 1 manager and 2 workers and both workers are in Drain state so i am only deploying services in the manager. I am trying to build a ELK stack from the ground up fully with docker containers so i can understand how every service works and interacts with each other. I managed to create the elasticsearch service and connect it to the Kibana service and bring some basic data (apparently Kibana creates data to monitor itself because some Kibana indexes were created), but i am having issues with Metricbeat because even when is up and getting metrics from the server (i am collecting metrics from the server and not the container) it doesn't create any index. Here are my service creation:

Elasticsearch

docker service create
--name es-master
-p 9200:9200
--network elk_stack
--env cluster.name=elk_cluster_dev
--env node.name=es01
--env discovery.type=single-node
--replicas 1
docker.elastic.co/elasticsearch/elasticsearch:7.3.2

Kibana

docker service create
--name kibana
-p 5601:5601
--mount type=bind,src=/home/repositories/ELK/kibana.yml,dst="${KIBANA_HOME}"/config/kibana.yml
--network elk_stack
--env elasticsearch.hosts=http://es-master:9200
--replicas 1
docker.elastic.co/kibana/kibana:7.3.2

Metricbeat

docker service create
--name metricbeat
--mount type=bind,src=/home/repositories/ELK/metricbeat.yml,dst=/usr/share/metricbeat/metricbeat.yml
--mount type=bind,src=/home/repositories/ELK/modules.d/,dst=/usr/share/metricbeat/modules.d/
--mount type=bind,src=/proc,dst=/hostfs/proc,readonly
--mount type=bind,src=/sys/fs/cgroup,dst=/hostfs/sys/fs/cgroup,readonly
--mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock
--mount type=bind,src=/,dst=/hostfs,readonly
--network elk_stack
--mode global
docker.elastic.co/beats/metricbeat:7.3.2

Configuration files

Kibana.yml

server.name: kibana
server.host: "0"
elasticsearch.hosts: [ "http://es-master:9200" ]
xpack.monitoring.ui.container.elasticsearch.enabled: true

metricbeat.yml

metricbeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.period: 10s
reload.enabled: false

setup.dashboards.enabled: true

output.elasticsearch:
hosts: ["es-master:9200"]

setup.kibana:
host: "http://kibana:5601"

modules.d/docker.yml

metricbeat.modules:

  • module: docker
    metricsets:
    • "container"
    • "cpu"
    • "diskio"
    • "event"
    • "healthcheck"
    • "info"
      #- "image"
    • "memory"
    • "network"
      hosts: ["unix:///var/run/docker.sock"]
      period: 10s
      enabled: true

Elasticsearch Index list (executed inside the metricbeat container)

docker exec -it e8071e22983c bash
bash-4.2$ curl -XGET 'es-master:9200/_cat/indices?v&pretty'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open .kibana Ot8A4987QaizJz_dZNdpIw 1 1 1 0 6.7kb 6.7kb

Do you what can be happening? I wil read some ELK stacks that are lying around in github uploaded by other admin, but i will appreciate any help that you can give me.

2

hmm do you see any error in metricbeat log?
Have you tried to run the setup command for Metricbeat? https://www.elastic.co/guide/en/beats/metricbeat/current/running-on-docker.html

docker.elastic.co/beats/metricbeat:7.3.2 \
setup -E setup.kibana.host=kibana:5601 \
-E output.elasticsearch.hosts=["elasticsearch:9200"]

If metricbeat is running and sending metrics to ES, kibana should have an index for it.

3

I actually made 2 additions to the service creation and it worked
Basically i added the --user=root and the -E output.elasticsearch.hosts=["elasticsearch:9200"] and the end. I forgot to post it here for others to see

docker service create
--name metricbeat
--user=root
--mount type=bind,src=/home/repositories/ELK/metricbeat/metricbeat.docker.yml,dst=/usr/share/metricbeat/metricbeat.yml
--mount type=bind,src=/home/repositories/ELK/modules.d/,dst=/usr/share/metricbeat/modules.d/
--mount type=bind,src=/proc,dst=/hostfs/proc,readonly
--mount type=bind,src=/sys/fs/cgroup,dst=/hostfs/sys/fs/cgroup,readonly
--mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock
--mount type=bind,src=/,dst=/hostfs,readonly
--network elk_stack
--mode global
docker.elastic.co/beats/metricbeat:7.3.2 -e
-E output.elasticsearch.hosts=["es-master:9200"]

Also, my new config file is like this

metricbeat.config:
modules:
path: ${path.config}/modules.d/*.yml
# Reload module configs as they change:
reload.enabled: false

metricbeat.autodiscover:
providers:
- type: docker
hints.enabled: true

metricbeat.modules:

  • module: docker
    metricsets:
    • "container"
    • "cpu"
    • "diskio"
    • "healthcheck"
    • "info"
      #- "image"
    • "memory"
    • "network"
      hosts: ["unix:///var/run/docker.sock"]
      period: 10s
      enabled: true

processors:

  • add_cloud_metadata: ~

setup.dashboards.enabled: true

output.elasticsearch:
hosts: ["es-master:9200"]

setup.kibana:
host: "http://kibana:5601"
I think that the autodiscover also did the trick

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.