Update Kibana for all visualisations to handle fields with null values

rbergmann · August 6, 2020, 9:37am

I understand that from v7+ there is an issue with the handling of null values in fields.
This would check it.
doc[<field>].size()==0

https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.html#_accessing_missing_document_values_will_throw_an_error

Could someone please let me know how I should update Kibana so that I don't encounter this issue when creating visualisations through the UI.
Thanks

thomasneirynck · August 6, 2020, 2:52pm

hi @rbergmann,

upgrade documentation is here Upgrade Kibana | Kibana Guide [8.11] | Elastic . That's probably the best way to start.

As for updating the scripts, you would need to add an if-check before actually accessing the field, and handle the condition explicitly.

if( doc['field'].size() == 0) { 
//...handle missing value....  
} else { 
//...happy path ... 
}

rbergmann · August 6, 2020, 3:19pm

Thanks @thomasneirynck for the quick response.
I get that. For users that create visualisations via the Kibana UI using the "clicky" option, how could this check be included to make sure all visualisation scripts include it?
Also, would you suggest we upgrade Kibana to the latest version? Would that fix this issue?

rbergmann · August 11, 2020, 5:53pm

@thomasneirynck could you please let us know if there is already a solution for this for Kibana UI visualisations and searches, not the scripted stuff you already answered.

thomasneirynck · August 11, 2020, 6:13pm

hi @rbergmann

For users that create visualisations via the Kibana UI using the "clicky" option, how could this check be included to make sure all visualisation scripts include it?

for the "clicky"-users nothing really changes in how they should use the UX.

It does mean that for users who have existing scripted fields in an index pattern (Index patterns has been renamed to data views. | Kibana Guide [8.11] | Elastic), you might have to add that if-check.

Also, would you suggest we upgrade Kibana to the latest version? Would that fix this issue?

It's more of the other way around. If you upgrade (e.g from 6.x to 7.x), you would need to inspect your existing scripted-fields and see if they need that additional check. So upgrading will require you to add that .size() condition if you're reading out values from a document.

rbergmann · August 11, 2020, 10:00pm

Thanks @thomasneirynck for that response. We are not using scripted fields. See attached screenshot.

rbergmann · August 12, 2020, 8:21am

Could it be a bug in Kibana 7.3.2? Would upgrading Kibana be an option worth trying?
Here is another screenshot showing not very descriptive error when trying to view data via Discover:

rbergmann · August 17, 2020, 7:00am

@thomasneirynck
if you have a solution for this Kibana error I would very much appreciate to hear about it.
Many thanks

rbergmann · August 20, 2020, 8:19am

@thomasneirynck I take it there is no fix for this Kibana bug yet?

rbergmann · August 26, 2020, 6:58am

@thomasneirynck - is there any solution for Kibana visualisations?

rbergmann · September 4, 2020, 8:39am

@thomasneirynck - we are looking at getting your professional support, have requested pricing information.

Just loading 40 days of indexes in Kibana Discover throws this not very useful error:

Any help here would be much appreciated.

system · October 2, 2020, 8:39am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.