Kibana and unknown field

Elastic Stack Kibana
#1

Hi there!

I created an Index Pattern ".monitoring-es*" and I try to create a dashboard on it.

I would like to use the field "index_stats.shards.total", like in the discover :
discover

Could you please help me?

  • Why my field is "unknown field"?
  • I can't see him in the Index Pattern
  • I can't change the index mapping

Best regards,

#2

It seems like this field is not indexed, so you can't build visulizations on top of it. Can you share the complete mapping of your monitoring index?

#3

Hello,

Thanks for your quick answer! The mapping is huge so I can't paste it here...

However, as you say, there is no index_stats.shards.total field in the mapping but why I can see it in discover if the field is not available?

Best regards,

#4

Hey, it works because Discover is operating on the _source object of the individual documents, so it is able to show field which are not indexed. Visualizations however rely on aggregations for performance reasons, and those only work on indexed fields.

#5

Thanks.

So on this specific index (directly created by Elasticsearch) I will not be able to use this field?

#6

The field is indexed, but it is not mapped, as it does not appear in the mappings.

If you want to run an aggregation on a field (which is usually necessary for building any kind of visualization), you'll need to create a custom mapping (using index templates) where you explicitly map this field.

1 Like
#7

Hi @chrisronline!

This is a very good idea but the index (.monitoring-es*) is not directly managed by me but by Elasticsearch itself.

I'm not sure I will be able to modify the mapping (to add a template) on this index?

Best regards,

#8

Hi @vincent2mots

You are able to add additional templates that affect indices that you don't manage directly.

Try putting this into Kibana dev tools:

PUT _template/custom_monitoring_template
{
  "index_patterns": [
    ".monitoring-es-*"
  ],
  "mappings": {
    "properties": {
      "index_stats": {
        "properties": {
          "shards": {
            "properties": {
              "total": {
                "type": "long"
              }
            }
          }
        }
      }
    }
  }
}

This will affect newly created indices, so it will not automatically start working until the next index is created (which will be tomorrow)

#9

Thanks!! That's perfect!

#10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.