Metricbeat Kibana dashboard not loading

[Keevin_Ace_Viray]

Hi,

I have setup Metricbeat(System module) -> Kafka -> Logstash -> Elasticsearch -> Kibana on our Linux local environment. I can see from the Discovery tab in Kibana that there is a lot of hits coming from Metricbeat, but once I loaded the generated Kibana dashboard for Metricbeat system module there are no data being loaded. Is there any additional config for this dashboard? Btw, my index is metricbeat-*

[shaunak (Shaunak Kashyap)]

Hi @Keevin_Ace_Viray, welcome to the Elastic community forums!

A couple of questions:

1. What version(s) of Metricbeat, Elasticsearch, and Kibana are you using?

2. Could you post the results of the following query please?

   POST metricbeat-*/_search?filter_path=hits.hits._source.system,hits.total
   {
     "query": {
       "bool": {
         "filter": {
           "term": {
             "event.module": "system"
           }
         }
       }
     },
     "collapse": {
       "field": "event.dataset"
     }
   }
   

Thanks,

Shaunak

[Keevin_Ace_Viray]

Hi @shaunak, thank you!

1. I'm using version 7.6.2
2. Please see below.
   
   

[Keevin_Ace_Viray]

Hi,

I'm using version 7.6.1. Please see screenshot below.

[shaunak (Shaunak Kashyap)]

Hmm, it looks like mappings might be missing. Could you please post the results of the following queries?

GET _cat/templates/metricbeat*
GET _template/metricbeat-*?filter_path=*.mappings
GET metricbeat-*/_mapping

The results of the last two queries might be large so feel free to post them on https://pastebin.com/ or http://gist.github.com/ first and then post the links here.

Thanks,

Shaunak

[Keevin_Ace_Viray]

Hi,

I only got response from the third query you provided.
https://pastebin.com/gkZAqWD5

[shaunak (Shaunak Kashyap)]

Thanks, that's useful. It tells us that for some reason the Metricbeat index templates did not get created. As a result, the Metricbeat data is being indexed in with default mappings as it comes in to Elasticsearch. These mappings are not what the dashboards expect, which would explain why they are blank.

@Keevin_Ace_Viray

I have setup Metricbeat(System module) -> Kafka -> Logstash -> Elasticsearch -> Kibana

This is probably why the Metricbeat templates didn't get loaded automatically. Whenever Metricbeat sends it's data to Elasticsearch indirectly — as is the case with your setup — Metricbeat doesn't have a direct connection to Elasticsearch. As such it cannot automatically load the index templates into Elasticsearch at startup. To account for such setups, Metricbeat has a setup command that must be run prior to starting Metricbeat for ingesting data. You can learn more about this over here: Load the Elasticsearch index template | Metricbeat Reference [8.11] | Elastic.

Shaunak

[Keevin_Ace_Viray]

I see, I was thinking if I should have connected Metricbeat directly to Elasticsearch first on the initial setup so that the templates will be loaded first. Then change it to my intended setup. Anyway, I will check first the link you provided.

[Keevin_Ace_Viray]

@shaunak

GET metricbeat-*/_mapping

Finally got it to work! Thank you so much @shaunak for the help!