Kibana 5.0.0: _missing_ is not working anymore

Elastic Stack Kibana
1

Hi,

When using Kibana 4.6 I used _missing_:@somefield to find documents with an empty value in @somefield. In Kibana 5.0.0 this doesn't work anymore. I can't find anywhere that this was changed or how to search for emty (keyword-)fields.
What is the proper way to do it now?

Also, in Kibana 4.6, visualizations on this field suppressed documents with an empty value in @somefield, so no empty buckets were shown on a vertical bar chart.
In Kibana 5.0.0 they show up in the visualization and I can find no way to suppress them again, meaning that my visualizations from 4.6 are different after upgrading to 5.0.

Is this changed standard behaviour?

Thanks,

Marten

2

Hi Marten,

NOT _exists_:@somefield

Unfortunately, I don't see where the removal of _missing_ is documented, but I do see in the Elasticsearch Query String Syntax documentation, Query string query | Elasticsearch Guide [8.11] | Elastic, that the _missing_ filter is documented for ES 2.x, but not for 5.0.

Seems like the removal should be documented here, but it isn't: Search and Query DSL changes | Elasticsearch Guide [5.0] | Elastic

3

Hi Tim,

Thanks for your answer.
Unfortunately your suggestion doesn't give the desired results also.
NOT _exists_@somefield returns documents which don't have the field.
I'm looking for a solution to search for documents which have the field but where the field is empty.

I've searched al breaking changes documents also but, like you, I couldn't find any documentation on this. Btw, _exists_ is also removed from the documentation.

The second part of my question however is a much bigger problem to me.
I thought it was standard for Kibana to suppress empty buckets in visualizations, but that doesn't seem to be the case anymore.
I have empty entries in my bar charts, in my tables etc.
Again, I can't find any documentation on this.

Marten

4

Search for "_exists_" on Query string query | Elasticsearch Guide [8.11] | Elastic

  • where the field title has any non-null value:
    _exists_:title

An empty string value is still a value, so to achieve what you want, those fields need to have an actual null value, or just be not present in the document.

If it's going to be a huge pain to re-index the data, you could try updating the fields using a script, or using scripted field with some logic that give a null if the original field is an empty string.

See Painless scripting language | Elasticsearch Guide [master] | Elastic

5

Sorry, my bad.
Completely overlooked it.

I'll do some more testing.
One index comes from SQL (with nulls), the other one from .csv
That's probably the reason.

Thanks Tim, this helped me a lot.