# Infrastructure UI is empty **URL:** \ **Category:** Metrics\ **Created:** [September 3, 2019, 7:36pm UTC](https://discuss.elastic.co/t/infrastructure-ui-is-empty/197933 "2019-09-03T19:36:41Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![luksi1](https://avatars.discourse-cdn.com/v4/letter/l/e68b1a/32.png) [@luksi1](https://discuss.elastic.co/u/luksi1)\ **Post date:** [September 3, 2019, 7:36pm UTC](https://discuss.elastic.co/t/infrastructure-ui-is-empty/197933/1 "2019-09-03T19:36:41Z")
Hello, I'm trying to diagnose why my Infrastructure UI is empty. The waffle map is empty, but the metrics explorer is populated. I can search the metricbeat-\* index without any problems. The problem arose when upgrading metricbeat to 7.x. Kibana + Elastissearch versions: 7.3.1 Everything goes straight to Elasticsearch. The Infrastructure UI configruration option is using "host.name" to identify hosts. And here is the mapping. ``` GET /metricbeat-*/_mapping/field/host.name { "metricbeat-7.3.1" : { "mappings" : { "host.name" : { "full_name" : "host.name", "mapping" : { "name" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } } } } } } } ``` ---
**Author:** ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)\ **Post date:** [September 3, 2019, 9:07pm UTC](https://discuss.elastic.co/t/infrastructure-ui-is-empty/197933/2 "2019-09-03T21:07:10Z")
Hi @luksi1, by looking at the mapping you're already on the right track. The fact that there is a sub-field called `host.name.keyword` indicates that the metricbeat index template is not applied correctly. Could you let us inspect the result of ```auto GET /_template/metricbeat-7.3.1 ``` ? ---
**Author:** ![luksi1](https://avatars.discourse-cdn.com/v4/letter/l/e68b1a/32.png) [@luksi1](https://discuss.elastic.co/u/luksi1)\ **Post date:** [September 4, 2019, 12:42am UTC](https://discuss.elastic.co/t/infrastructure-ui-is-empty/197933/3 "2019-09-04T00:42:46Z")
Unfortunately I've already tried setting the host field to host.name.keyword. I tried again and I'm getting the same results. I actually removed the template and removed the index to see if this was the problem, and nothing. For the sake of completeness, I've restarted an agent to create the template and removed the metricbeat-\* index and I'm still experiencing the same thing. Once again, I'm still getting nothing in the waffle map and plenty of data in the metrics view. Here's the template (I couldn't post it here due to length): > ---
**Author:** ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)\ **Post date:** [September 4, 2019, 8:36am UTC](https://discuss.elastic.co/t/infrastructure-ui-is-empty/197933/4 "2019-09-04T08:36:03Z")
> Unfortunately I've already tried setting the host field to host.name.keyword. Yes, it's not as easy as that because the metric calculations rely on other fields as well. I was just using this one field as an indicator that the mapping is incorrect in general. So from the index template you posted we can see that it matches indices with names like `metricbeat-7.3.1-*`. From your initial post I learned that your index is called `metricbeat-7.3.1`, which does not match that wildcard string. The template is therefore not applied and Elasticsearch's incorrect dynamic mapping kicks in. By default the beat indices are partitioned by day or using a increasing counter (e.g. `metricbeat-7.3.1-2019.09.04` or `metricbeat-7.3.1-1`) to enable easy management of retention times and to increase query performance. I would recommend to stick with that, which should also lead to the correct index template being applied. ---
**Author:** ![luksi1](https://avatars.discourse-cdn.com/v4/letter/l/e68b1a/32.png) [@luksi1](https://discuss.elastic.co/u/luksi1)\ **Post date:** [September 4, 2019, 12:01pm UTC](https://discuss.elastic.co/t/infrastructure-ui-is-empty/197933/5 "2019-09-04T12:01:47Z")
Well, that would make sense. But I am allowing metricbeat to create the default index, which is metricbeat-7.3.1. I'm not specifying an index in output.elasticsearch. It's simply not tagging them by default with a date or a sequential number. I can add it to our fleet of metricbeat agents via config management, but it just doesn't feel like I should have to. ---
**Author:** ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)\ **Post date:** [September 4, 2019, 2:13pm UTC](https://discuss.elastic.co/t/infrastructure-ui-is-empty/197933/6 "2019-09-04T14:13:50Z")
The default setting should indeed produce compatible index names. 🤔 Would it be possible to provide the filebeat configuration (without credentials, of course)? ---
**Author:** ![luksi1](https://avatars.discourse-cdn.com/v4/letter/l/e68b1a/32.png) [@luksi1](https://discuss.elastic.co/u/luksi1)\ **Post date:** [September 4, 2019, 2:24pm UTC](https://discuss.elastic.co/t/infrastructure-ui-is-empty/197933/7 "2019-09-04T14:24:51Z")
Filebeat or Metricbeat? ---
**Author:** ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)\ **Post date:** [September 4, 2019, 2:59pm UTC](https://discuss.elastic.co/t/infrastructure-ui-is-empty/197933/8 "2019-09-04T14:59:45Z")
Metricbeat 😇 sorry, I mixed it up with another thread ---
**Author:** ![luksi1](https://avatars.discourse-cdn.com/v4/letter/l/e68b1a/32.png) [@luksi1](https://discuss.elastic.co/u/luksi1)\ **Post date:** [September 4, 2019, 3:27pm UTC](https://discuss.elastic.co/t/infrastructure-ui-is-empty/197933/9 "2019-09-04T15:27:29Z")
I think there's an ILM thing going on here. Here's the metricbeat.yml configuration: ``` metricbeat.config.modules: path: ${path.config}/modules.d/*.yml reload.enabled: true setup.template.settings: index.number_of_shards: 1 index.codec: best_compression setup.ilm.enabled: auto setup.kibana: host: "https://kibana.domain.se:443" output.elasticsearch: hosts: ['foo.domain.se:9200', 'bar.domain.se:9200', 'baz.domain.se:9200', 'boo.domain.se:9200', 'bah.domain.se:9200'] protocol: "https" username: "elastic.beats" password: "my.password" ssl.certificate_authorities: ['/etc/metricbeat/certs/ca.pem'] ssl.certificate: "/etc/metricbeat/certs/foo.domain.se.crt" ssl.key: "/etc/metricbeat/certs/foo.domain.se.key" xpack.monitoring: enabled: true elasticsearch: username: "elastic.beats" password: "my.password" ``` ---
**Author:** ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)\ **Post date:** [September 4, 2019, 5:28pm UTC](https://discuss.elastic.co/t/infrastructure-ui-is-empty/197933/10 "2019-09-04T17:28:10Z")
If ILM was used `metricbeat-7.3.1` should be an alias with the actual index being something like `metricbeat-7.3.1-2019.09.04-000001`. Can you check that the ILM policy referred to in the index template is installed? ```auto GET _ilm/policy/metricbeat-7.3.1 ``` ---
**Author:** ![luksi1](https://avatars.discourse-cdn.com/v4/letter/l/e68b1a/32.png) [@luksi1](https://discuss.elastic.co/u/luksi1)\ **Post date:** [September 4, 2019, 5:39pm UTC](https://discuss.elastic.co/t/infrastructure-ui-is-empty/197933/11 "2019-09-04T17:39:34Z")
``` { "metricbeat-7.3.1" : { "version" : 4, "modified_date" : "2019-08-30T21:05:24.199Z", "policy" : { "phases" : { "hot" : { "min_age" : "0ms", "actions" : { "rollover" : { "max_size" : "50gb", "max_age" : "30d" } } } } } } } ``` ---
**Author:** ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)\ **Post date:** [September 4, 2019, 5:51pm UTC](https://discuss.elastic.co/t/infrastructure-ui-is-empty/197933/12 "2019-09-04T17:51:46Z")
That ILM policy looks good. So my hypothesis is: - someone or something created an index `metricbeat-7.3.1`, which doesn't get the correct mapping because it doesn't match any index template - when starting metricbeat it can't correctly set up the `metricbeat-7.3.1` rollover alias and just uses the existing index (which doesn't have the mapping) To confirm, can you check that on startup metricbeat prints something like ```auto INFO [index-management] idxmgmt/std.go:298 Write alias exists already ``` ? ---
**Author:** ![luksi1](https://avatars.discourse-cdn.com/v4/letter/l/e68b1a/32.png) [@luksi1](https://discuss.elastic.co/u/luksi1)\ **Post date:** [September 4, 2019, 6:50pm UTC](https://discuss.elastic.co/t/infrastructure-ui-is-empty/197933/13 "2019-09-04T18:50:09Z")
How did you know? 🙂 `Sep 4 20:46:29 vgas1607 metricbeat[17620]: 2019-09-04T20:46:29.394+0200#011INFO#011[index-management]#011idxmgmt/std.go:298#011Write alias exists already` Since you're on the money, how do I resolve this? We have a few nodes writing to that index constantly? 😕 So even if I remove this index, all the nodes hit that index immediately, not allowing me to the restart the node fast enough to create the alias and the respective indices. ---
**Author:** ![luksi1](https://avatars.discourse-cdn.com/v4/letter/l/e68b1a/32.png) [@luksi1](https://discuss.elastic.co/u/luksi1)\ **Post date:** [September 4, 2019, 7:54pm UTC](https://discuss.elastic.co/t/infrastructure-ui-is-empty/197933/14 "2019-09-04T19:54:43Z")
I actually resolved it by stopping all of our agents, restarting one, and then restarting everything up. Not ideal, but it got everything going. Thanks a lot for your time. ---
**Author:** ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)\ **Post date:** [September 4, 2019, 9:17pm UTC](https://discuss.elastic.co/t/infrastructure-ui-is-empty/197933/15 "2019-09-04T21:17:41Z")
Glad to hear you could resolve it. With 7.4 the beats will actually check for that condition so startup will fail with a corresponding message: [https://github.com/elastic/beats/pull/13070](https://github.com/elastic/beats/pull/13070) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 2, 2019, 9:17pm UTC](https://discuss.elastic.co/t/infrastructure-ui-is-empty/197933/16 "2019-10-02T21:17:47Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.