Field.keyword is empty, yet something is shown under field

[Ebisu]

My field msg is a result of using grok with %{GREEDYDATA}. If I apply the following filter in Kibana's "Disover": NOT msg.keyword: "exists", roughly half of all events match. But for those msg is not empty. Can somebody explain to me what is going on here?

[Brandon_Kobel]

Hey @Ebisu, when using the query NOT msg.keyword: "exists" it will return all documents which don't have the string "exists" for the msg.keyword field.

To better explain this behavior, if you insert the following 3 documents into ES using Dev Tools:

POST ebisu/_doc
{
  "msg": "exists"
}

POST ebisu/_doc
{
  "msg": "foo"
}

POST ebisu/_doc
{
  "msg": "bar"
}

If we use the aforementioned filter in the query bar, we'll see both "foo" and "bar" returned, as they don't have a msg.keyword field of "exists":

[Ebisu]

Thanks for your answer.

If I paste NOT msg.keyword: "exists" in the filter field and click refresh, I get a different number of matches than if I click on "Add filter", select msg.keyword, select "does not exist" and click on "save".

[Brandon_Kobel]

@Ebisu, that is to be expected. The query-bar and filter-bar use different syntax per https://www.elastic.co/guide/en/kibana/current/search.html

[Ebisu]

Yes, I meant the query-bar syntax, sorry! How is it possible that Kibana claims field.keyword doesn't exist, while field is not empty?

If I make a pie chart Count / Term>field.keyword and include missing values, I get a huge slice for "missing".

[Ebisu]

I found the reason: some entries were too long and Elasticsearch choose “text” instead of “keyword”. Sorry, I'm new to this – it wasn't really a Kibana question.