Filter dropdown for a field missing one of the top 5 values displayed under "Available fields" section

Elastic Stack Kibana
1

I am having an issue with the dropdown values listed for a field when trying to filter for an index. It always shows the same 10 values but doesn't show one of the top 5 values listed in the "Available fields" section for the same field

2

Hi @deepalalitaakella,

thanks for your question. If you open the filter editor it's just showing the first 10 entries it finds, but once you start typing it should narrow down the search and query the server for more possible values matching the already entered prefix. Do the values from the "Available fields" section show up once you start typing them?

If you don't get your desired values even if you are starting to type the prefix, it's possible that you have a lot of data and the server is running into a timeout. If that should be the case there is a setting starting 7.3 kibana.autocompleteTimeout and kibana.autocompleteTerminateAfter that can be relaxed a bit to get a more complete picture. (https://www.elastic.co/guide/en/kibana/current/settings.html)

3

Thanks for the Reply Joe . No it doesn't show up the other values in the filter when I start typing in

4

We are using Kibana 7.2.0 . So are the features kibana.autocompleteTimeout and kibana.autocompleteTerminateAfter available for this version?

5

Unfortunately the setting is only available in Kibana 7.3 and upcoming versions

6

Thanks for the help Joe!!!

7

Another possibility - did you start typing the full value from the start or tried to match a substring in the middle of the value? Depending on how you indexed your data you have to match the value perfectly from the start - e.g. if the value My partial string is indexed as a keyword and I start typing partial in the dropdown, it won't show up because the field is not analyzed on a word-per-word basis

8

I did type in full word since it just a 4 letter Value.

9

I was looking at another aspect to this wrt the filebeat configuration where the fields.type might be having a wrong check here and also there is no symbolic link to the log file path mentioned here.

filebeat.yml
filebeat.inputs:

  • type: log
    enabled: true
    fields.type: error_log
    symlinks: true
    paths:
    • /u/logs/docker/laravel/laravel*.log
10

Do you think your data didn't get indexed correctly? You can check whether it contains the expected data by looking at the documents in Discover or by sending queries to your Elasticsearch cluster directly.

11

Yes the data seems to have indexed correctly because the same field with .keyword as the extension contains the desired filter value.

i.e.
app is a field that doesn't show the value "SBFE" in the filter drop down but
app.keyword does show the same value.

12

A field has to be indexed as a keyword field in the mapping of the index (https://www.elastic.co/guide/en/elasticsearch/reference/current/keyword.html) to provide a filter dropdown. Otherwise it should display a regular input field:

Field indexed as keyword:

19
Screenshot 2019-08-19 at 06.15.19.png800×324 12.7 KB

Field indexed as text:

10
Screenshot 2019-08-19 at 06.15.10.png800×428 15.4 KB

Make sure to use the keyword variant if you want to look at a list of all terms of a field, ideally by looking at the mapping of the index: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-get-mapping.html

13

But the issue is both the fields here i.e. app and app.keyword show the filter dropdown but the app field is missing out some important filter values while app.keyword is not.
What I am unable to understand is why is there a difference in the filter dropdown values for both when they are extracted from the same source field.

14

Maybe there is something strange configured in the mapping. Could you run GET /<YOUR_INDEX_NAME>/_mapping in the dev console and post the result here?

15 
    "app" : {
      "type" : "text",
      "norms" : false,
      "fields" : {
        "keyword" : {
          "type" : "keyword",
          "ignore_above" : 256
        }
      }
    },

    "fields" : {
      "properties" : {
        "app" : {
          "type" : "keyword"
        },
        "env" : {
          "type" : "keyword"
        },
        "infra_group" : {
          "type" : "keyword"
        },
        "loc" : {
          "type" : "keyword"
        },
        "type" : {
          "type" : "keyword"
        },
        "type2" : {
          "type" : "keyword"
        }
      }
    },

There are a lot of fields so just updating for the app field which is having an issue

16

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.