Metricbeat can't collect elasticsearch metrics

[lisiyu (Angela)]

The elasticsearch module is enabled in metricbeat. But metricbeat index have no data for elastic metrics.

Version

• Metricbeat: 7.13.1
• Elasticsearch: 7.13.1
• Kibana: 7.13.1

Config

• Metricbeat:

......
    metricbeat.config.modules:
      # Mounted `metricbeat-daemonset-modules` configmap:
      path: /usr/share/metricbeat/modules.d/*.yml
      # Reload module configs as they change:
      reload.enabled: true
      setup.dashboards.enabled: true
      setup.kibana.host: "http://kibana-svc:80"
......
  elasticsearch.yml: |-
    - module: elasticsearch
      xpack.enabled: true
      period: 30s
      hosts: ["http://elastic-svc:9200"]
      metricsets:
        - node
        - node_stats
        - index
        - index_recovery
        - index_summary
        - shard

• ElasticSearch

bootstrap.memory_lock: true
node.name: ${HOSTNAME}

action.destructive_requires_name: true
indices.fielddata.cache.size: 1% # default is unbounded
cluster.name: es-cluster

# only data nodes should have ingest and http capabilities
node.master: false
node.data: false
node.ingest: false
xpack.security.enabled: false
xpack.monitoring.enabled: true
xpack.monitoring.collection.enabled: true
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch

• Kibana
  
  
  
  
  

[afgomez (Alejandro Fernández Gómez)]

Hi @lisiyu,

Could you check if the .monitoring-es* indices exist and have data? In the the Kibana developer tools, run the following query.

GET /_cat/indices/.monitoring-es*?v

Secondly, in the documentation mentions that the elasticsearch module should not have any metricsets. Can you try commenting that block in your modules/elasticsearch.yml file? Alternatively, try using the elasticsearch-xpack module instead of the elasticsearch module.

Let me know if any of this works.

[lisiyu (Angela)]

hello @afgomez!
I can see index of .monitoring-es*. But I can't discover it .

You mean like this?

  elasticsearch-xpack.yml: |-
    - module: elasticsearch-xpack
      xpack.enabled: true
      period: 30s
      hosts: ["http://elastic-svc:9200"]
      metricsets:
        - node
        - node_stats
        - index
        - index_recovery
        - index_summary
        - shard

[afgomez (Alejandro Fernández Gómez)]

I'm sorry, I didn't understand correctly the first time what you wanted to achieve. The elastic-stack-monitoring tag threw me off track.

You want to get elasticsearch metrics inside the metricbeat-* indices, so you can see them in Discover, correct?

For that you need to remove the x-pack.enabled: true line from your modules/elasticsearch.yml.

What that setting does is tell Metricbeat to collect the data in .monitoring-es*. Those indices are used by the Stack monitoring app in kibana.

[lisiyu (Angela)]

I removed x-pack.enabled: true. It working. Thank you .