Undefined fields ARE SHOWN in "Available fields" in Kibana

[Rudolf_Reddy_Macejka]

Hello,

I would like to try to understand and fix "cosmetic" issue in Discovery panel in Kibana.

I have a mapping by which I wanted to allow store any field, but index only defined fields. Hence I have chosen dynamic:false. I do not want to use strict, because undefined fields will not be stored. :

 "mappings": {
    "doc": {
      "dynamic": false,
      "numeric_detection": true,
      "properties": {
        "@timestamp": {"type": "date"},
        "type": {"type":"keyword", "ignore_above": 10},
        "sourceSystem": {"type":"keyword", "ignore_above": 20},

If I will POST a message exact as is the mapping, all is OK.
But If I added an undefined field, it was stored, but the undefined field appear in Discovery view in "Available fields". Even with question-mark, because it is not defined in mapping.
Fortunately in index pattern, either after refresh, the undefined field was not added. So this part works.

But why it allows to use the field in Discovery? Even if the field should not be indexed so should not be searchable or aggregatable.

How to hide fields in Discovery Available fields which are not defined in mapping?

[matw (Matthias Wilhelm)]

So what Discover currently does is merging the fields of your mapping with the ones provided by the _source of your records. That's why those fields show up and are selectable in Discover's sidebar. You could exclude those fields in the index pattern management with source filters

And we're currently discussing to change that behavior, you could follow this issue to see progress, if you have a GitHub account

[Discover] Switch from _source to fields when fetching data

Best,
Matthias