# Kibana and unknown field

**URL:** https://discuss.elastic.co/t/kibana-and-unknown-field/261778
**Category:** Kibana
**Tags:** elastic-stack-monitoring
**Created:** [January 21, 2021, 12:21pm UTC](https://discuss.elastic.co/t/kibana-and-unknown-field/261778 "2021-01-21T12:21:00Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![vincent2mots](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vincent2mots/32/55226_2.png) [@vincent2mots](https://discuss.elastic.co/u/vincent2mots)
#### Post date: [January 21, 2021, 12:21pm UTC](https://discuss.elastic.co/t/kibana-and-unknown-field/261778/1 "2021-01-21T12:21:00Z")

</div>

Hi there!

I created an Index Pattern ".monitoring-es\*" and I try to create a dashboard on it.

I would like to use the field "index\_stats.shards.total", like in the discover :  
 ![discover](https://us1.discourse-cdn.com/elastic/original/3X/0/6/0647c58501ab3c45755a2bbc05109b0e140638df.jpeg)

Could you please help me?

- Why my field is "unknown field"?
- I can't see him in the Index Pattern
- I can't change the index mapping

Best regards,

---

<div class="post-metadata">

### Author: ![flash1293](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/flash1293/32/41227_2.png) [@flash1293](https://discuss.elastic.co/u/flash1293)
#### Post date: [January 21, 2021, 12:23pm UTC](https://discuss.elastic.co/t/kibana-and-unknown-field/261778/2 "2021-01-21T12:23:39Z")

</div>

It seems like this field is not indexed, so you can't build visulizations on top of it. Can you share the complete mapping of your monitoring index?

---

<div class="post-metadata">

### Author: ![vincent2mots](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vincent2mots/32/55226_2.png) [@vincent2mots](https://discuss.elastic.co/u/vincent2mots)
#### Post date: [January 21, 2021, 12:33pm UTC](https://discuss.elastic.co/t/kibana-and-unknown-field/261778/3 "2021-01-21T12:33:57Z")

</div>

Hello,

Thanks for your quick answer! The mapping is huge so I can't paste it here...

However, as you say, there is no index\_stats.shards.total field in the mapping but why I can see it in discover if the field is not available?

Best regards,

---

<div class="post-metadata">

### Author: ![flash1293](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/flash1293/32/41227_2.png) [@flash1293](https://discuss.elastic.co/u/flash1293)
#### Post date: [January 21, 2021, 12:38pm UTC](https://discuss.elastic.co/t/kibana-and-unknown-field/261778/4 "2021-01-21T12:38:18Z")

</div>

Hey, it works because Discover is operating on the `_source` object of the individual documents, so it is able to show field which are not indexed. Visualizations however rely on aggregations for performance reasons, and those only work on indexed fields.

---

<div class="post-metadata">

### Author: ![vincent2mots](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vincent2mots/32/55226_2.png) [@vincent2mots](https://discuss.elastic.co/u/vincent2mots)
#### Post date: [January 21, 2021, 1:10pm UTC](https://discuss.elastic.co/t/kibana-and-unknown-field/261778/5 "2021-01-21T13:10:56Z")

</div>

Thanks.

So on this specific index (directly created by Elasticsearch) I will not be able to use this field?

---

<div class="post-metadata">

### Author: ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)
#### Post date: [January 21, 2021, 1:59pm UTC](https://discuss.elastic.co/t/kibana-and-unknown-field/261778/6 "2021-01-21T13:59:54Z")

</div>

The field is indexed, but it is not mapped, as it does not appear in the [mappings](https://github.com/elastic/elasticsearch/blob/master/x-pack/plugin/core/src/main/resources/monitoring-es.json#L163).

If you want to run an aggregation on a field (which is usually necessary for building any kind of visualization), you'll need to create a custom mapping (using [index templates](https://www.elastic.co/guide/en/elasticsearch/reference/7.10/index-templates.html)) where you explicitly map this field.

---

<div class="post-metadata">

### Author: ![vincent2mots](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vincent2mots/32/55226_2.png) [@vincent2mots](https://discuss.elastic.co/u/vincent2mots)
#### Post date: [January 22, 2021, 7:55am UTC](https://discuss.elastic.co/t/kibana-and-unknown-field/261778/7 "2021-01-22T07:55:45Z")

</div>

Hi @chrisronline!

This is a very good idea but the index (.monitoring-es\*) is not directly managed by me but by Elasticsearch itself.

I'm not sure I will be able to modify the mapping (to add a template) on this index?

Best regards,

---

<div class="post-metadata">

### Author: ![chrisronline](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrisronline/32/28230_2.png) [@chrisronline](https://discuss.elastic.co/u/chrisronline)
#### Post date: [January 22, 2021, 2:20pm UTC](https://discuss.elastic.co/t/kibana-and-unknown-field/261778/8 "2021-01-22T14:20:30Z")

</div>

Hi @vincent2mots

You are able to add additional templates that affect indices that you don't manage directly.

Try putting this into Kibana dev tools:

```auto
PUT _template/custom_monitoring_template
{
  "index_patterns": [
    ".monitoring-es-*"
  ],
  "mappings": {
    "properties": {
      "index_stats": {
        "properties": {
          "shards": {
            "properties": {
              "total": {
                "type": "long"
              }
            }
          }
        }
      }
    }
  }
}

```

This will affect _newly created_ indices, so it will not automatically start working until the next index is created (which will be tomorrow)

---

<div class="post-metadata">

### Author: ![vincent2mots](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vincent2mots/32/55226_2.png) [@vincent2mots](https://discuss.elastic.co/u/vincent2mots)
#### Post date: [January 22, 2021, 2:58pm UTC](https://discuss.elastic.co/t/kibana-and-unknown-field/261778/9 "2021-01-22T14:58:49Z")

</div>

> [@chrisronline](#):
>
> ```auto
> PUT _template/custom_monitoring_template
> {
> "index_patterns": [
> ".monitoring-es-*"
> ],
> "mappings": {
> "properties": {
> "index_stats": {
> "properties": {
> "shards": {
> "properties": {
> "total": {
> "type": "long"
> }
> }
> }
> }
> }
> }
> }
> }
> 
> ```

Thanks!! That's perfect!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 19, 2021, 2:58pm UTC](https://discuss.elastic.co/t/kibana-and-unknown-field/261778/10 "2021-02-19T14:58:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
