# Where is my infrastructure data?

**URL:** https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039
**Category:** Metrics
**Created:** [November 16, 2018, 11:36am UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039 "2018-11-16T11:36:58Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![ethrbunny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ethrbunny/32/34603_2.png) [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)
#### Post date: [November 16, 2018, 11:36am UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/1 "2018-11-16T11:36:58Z")

</div>

Just discovered the new "infrastructure" menu item. Unfortunately, despite having many, many GB of data there doesn't seem to be anything to display here. I have plenty of dashboard data and am gathering from a slew of different \*beat providers.

I've tried a variety of options in the "search for infrastructure data" field.

So - what am I doing wrong?

---

<div class="post-metadata">

### Author: ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)
#### Post date: [November 16, 2018, 12:30pm UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/2 "2018-11-16T12:30:27Z")

</div>

Hi @ethrbunny,

Infrastructure UI relies on Metricbeat data, many of the metrics we show are available starting with  
Metricbeat 6.5 so I would recommend using it. For instance, we use information from [add\_host\_metadata](https://www.elastic.co/guide/en/beats/filebeat/master/add-host-metadata.html) to group information by host.

Best regards

---

<div class="post-metadata">

### Author: ![rschirin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rschirin/32/45283_2.png) [@rschirin](https://discuss.elastic.co/u/rschirin)
#### Post date: [November 16, 2018, 6:22pm UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/3 "2018-11-16T18:22:58Z")

</div>

so it's important to use Metricbeat 6.5? I was collecting data with Docker module on Metricbeat 6.4; Have I to update it?  
[here-infrastructure-UI](https://www.elastic.co/guide/en/infrastructure/guide/6.5/install-infrastructure-monitoring.html#_which_modules_and_configuration_options_do_i_enable) I saw a list of points to be enabled to collect data from Docker; is it really a list or can I just enable one of them?

---

<div class="post-metadata">

### Author: ![ethrbunny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ethrbunny/32/34603_2.png) [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)
#### Post date: [November 16, 2018, 7:00pm UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/4 "2018-11-16T19:00:35Z")

</div>

All my hosts are on puppet and set to upgrade whenever a new version is released. They should all be on 6.5 if that's what's available.

Is there something I can check to see what's going on?

---

<div class="post-metadata">

### Author: ![ethrbunny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ethrbunny/32/34603_2.png) [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)
#### Post date: [November 17, 2018, 1:43pm UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/5 "2018-11-17T13:43:51Z")

</div>

FWIW: Ive checked to make sure that [metric|file]beat on my various hosts are all running 6.5.

What else should I look for?

---

<div class="post-metadata">

### Author: ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)
#### Post date: [November 18, 2018, 11:29pm UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/6 "2018-11-18T23:29:42Z")

</div>

Hi again,

There are some things you can check to troubleshoot this issue:

- Are you getting metricbeat data in? You should be able to see it in the Discover page, for instance
- Are you using an index name different from the default? Infrastructure UI queries `metricbeat-*`

---

<div class="post-metadata">

### Author: ![rschirin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rschirin/32/45283_2.png) [@rschirin](https://discuss.elastic.co/u/rschirin)
#### Post date: [November 19, 2018, 1:33pm UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/7 "2018-11-19T13:33:35Z")

</div>

hi @exekias,  
is there a way to change that index\_pattern to check my index and not the default one?

---

<div class="post-metadata">

### Author: ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)
#### Post date: [November 19, 2018, 2:53pm UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/8 "2018-11-19T14:53:34Z")

</div>

Hi @rschirin,

you can change the index pattern used to read the metrics by changing the `xpack.infra.sources.default.metricAlias` setting in the kibana config file. Contrary to the name, it does not have to be an alias, any [index pattern supported by Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-index.html) should work.

---

<div class="post-metadata">

### Author: ![ethrbunny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ethrbunny/32/34603_2.png) [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)
#### Post date: [November 20, 2018, 2:54am UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/9 "2018-11-20T02:54:30Z")

</div>

Im definitely getting metricbeat-\* data

---

<div class="post-metadata">

### Author: ![niklaskurvinen](https://avatars.discourse-cdn.com/v4/letter/n/cdc98d/32.png) [@niklaskurvinen](https://discuss.elastic.co/u/niklaskurvinen)
#### Post date: [November 23, 2018, 7:41am UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/10 "2018-11-23T07:41:47Z")

</div>

Hi,

I'm seeing the same issue - infrastructure UI isn't populated ("There is no data to display.") even though I have metricbeat-\* data sent from 6.5.x using the system module.

Something i noticed is that the POST requests to "graphql" return an error:

`..."errors":[{"message":"[illegal_argument_exception] Fielddata is disabled on text fields by default. Set fielddata=true on [beat.hostname] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."...`

Anyone found a solution worth sharing?

---

<div class="post-metadata">

### Author: ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)
#### Post date: [November 23, 2018, 12:57pm UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/11 "2018-11-23T12:57:08Z")

</div>

Hi @niklaskurvinen,

that suggests a problem with the metricbeat mappings. Could you give us a sample of the output when querying

`/metricbeat-*/_mapping/doc/field/beat.hostname`

on your Elasticsearch API?

---

<div class="post-metadata">

### Author: ![niklaskurvinen](https://avatars.discourse-cdn.com/v4/letter/n/cdc98d/32.png) [@niklaskurvinen](https://discuss.elastic.co/u/niklaskurvinen)
#### Post date: [November 23, 2018, 1:45pm UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/12 "2018-11-23T13:45:38Z")

</div>

Hi @weltenwort! Sure thing!

`"metricbeat-2018.11.23" : { "mappings" : { "doc" : { "beat.hostname" : { "full_name" : "beat.hostname", "mapping" : { "hostname" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } } } } } }`

---

<div class="post-metadata">

### Author: ![niklaskurvinen](https://avatars.discourse-cdn.com/v4/letter/n/cdc98d/32.png) [@niklaskurvinen](https://discuss.elastic.co/u/niklaskurvinen)
#### Post date: [November 23, 2018, 2:43pm UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/13 "2018-11-23T14:43:24Z")

</div>

Oh! I changed on of my nodes to not use logstash and installed the index template (using `metricbeat setup`) and _that_ index is immediately picked up by the infrastructure UI.

So, now I just have to figure out how I can get logstash to use the same mapping as the index template - any tips on how to do it?

---

<div class="post-metadata">

### Author: ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)
#### Post date: [November 23, 2018, 3:02pm UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/14 "2018-11-23T15:02:50Z")

</div>

The index template by default matches `metricbeat-${VERSION}-*`. I assume your logstash configuration writes to different index names? You could pick metricbeat's template from the `_template/metricbeat-*` API response and write it back to Elasticsearch under a different name with an index pattern that matches your logstash configuration. See [the Elasticsearch docs on index templates](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html) for more usage examples of the `_template` API.

---

<div class="post-metadata">

### Author: ![niklaskurvinen](https://avatars.discourse-cdn.com/v4/letter/n/cdc98d/32.png) [@niklaskurvinen](https://discuss.elastic.co/u/niklaskurvinen)
#### Post date: [November 23, 2018, 3:11pm UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/15 "2018-11-23T15:11:45Z")

</div>

Cheers @weltenwort, that wasn't too hard!

Everything works as expected for me now. Thanks!

---

<div class="post-metadata">

### Author: ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)
#### Post date: [November 23, 2018, 3:14pm UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/16 "2018-11-23T15:14:28Z")

</div>

@niklaskurvinen, glad to hear it 👍

@ethrbunny, is it possible your problems are of similar origin?

---

<div class="post-metadata">

### Author: ![ethrbunny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ethrbunny/32/34603_2.png) [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)
#### Post date: [November 23, 2018, 11:13pm UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/17 "2018-11-23T23:13:00Z")

</div>

Yes that's a possibility. I use metricbeat-\*.

```
GET /metricbeat-*/_mapping/doc/field/beat.hostname

 "metricbeat-2018.11.23" : {
 "mappings" : {
  "doc" : {
    "beat.hostname" : {
      "full_name" : "beat.hostname",
      "mapping" : {
        "hostname" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    }
  }
}

```

Is there no way to change the infrastructure lookup to use my index?

---

<div class="post-metadata">

### Author: ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)
#### Post date: [November 26, 2018, 12:47pm UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/18 "2018-11-26T12:47:32Z")

</div>

You should be able to adapt the field to your index structure by setting

```auto
xpack.infra:
  sources:
    default:
      fields:
        hostname: 'beat.hostname.keyword'
        # pod: 'kubernetes.pod.name'
        # container: 'docker.container.name'

```

Depending in your indices you might also want to change the `pod` and `container` field settings I commented out above.

---

<div class="post-metadata">

### Author: ![ethrbunny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ethrbunny/32/34603_2.png) [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)
#### Post date: [November 27, 2018, 12:45pm UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/19 "2018-11-27T12:45:40Z")

</div>

err.. ok.. how? Where are those values set?

---

<div class="post-metadata">

### Author: ![weltenwort](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/weltenwort/32/53885_2.png) [@weltenwort](https://discuss.elastic.co/u/weltenwort)
#### Post date: [November 28, 2018, 6:24pm UTC](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039/20 "2018-11-28T18:24:37Z")

</div>

Sorry, in the Kibana config file (usually called `kibana.yml`). The exact location of that file depends on the installation method.

[Next page](https://discuss.elastic.co/t/where-is-my-infrastructure-data/157039.md?page=2)
