Update Kibana for all visualisations to handle fields with null values

[rbergmann (Robert Bergmann)]

I understand that from v7+ there is an issue with the handling of null values in fields.
This would check it.
doc[<field>].size()==0

https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.html#_accessing_missing_document_values_will_throw_an_error

Could someone please let me know how I should update Kibana so that I don't encounter this issue when creating visualisations through the UI.
Thanks

[thomasneirynck]

hi @rbergmann,

@rbergmann

how I should update Kibana

upgrade documentation is here Upgrade Kibana | Kibana Guide [8.11] | Elastic . That's probably the best way to start.

As for updating the scripts, you would need to add an if-check before actually accessing the field, and handle the condition explicitly.

if( doc['field'].size() == 0) { 
//...handle missing value....  
} else { 
//...happy path ... 
}

[rbergmann (Robert Bergmann)]

Thanks @thomasneirynck for the quick response.
I get that. For users that create visualisations via the Kibana UI using the "clicky" option, how could this check be included to make sure all visualisation scripts include it?
Also, would you suggest we upgrade Kibana to the latest version? Would that fix this issue?

[rbergmann (Robert Bergmann)]

@thomasneirynck could you please let us know if there is already a solution for this for Kibana UI visualisations and searches, not the scripted stuff you already answered.

[thomasneirynck]

hi @rbergmann

For users that create visualisations via the Kibana UI using the "clicky" option, how could this check be included to make sure all visualisation scripts include it?

for the "clicky"-users nothing really changes in how they should use the UX.

It does mean that for users who have existing scripted fields in an index pattern (Index patterns has been renamed to data views. | Kibana Guide [8.11] | Elastic), you might have to add that if-check.

Also, would you suggest we upgrade Kibana to the latest version? Would that fix this issue?

It's more of the other way around. If you upgrade (e.g from 6.x to 7.x), you would need to inspect your existing scripted-fields and see if they need that additional check. So upgrading will require you to add that .size() condition if you're reading out values from a document.

[rbergmann (Robert Bergmann)]

Thanks @thomasneirynck for that response. We are not using scripted fields. See attached screenshot.

[rbergmann (Robert Bergmann)]

Could it be a bug in Kibana 7.3.2? Would upgrading Kibana be an option worth trying?
Here is another screenshot showing not very descriptive error when trying to view data via Discover:

[rbergmann (Robert Bergmann)]

@thomasneirynck
if you have a solution for this Kibana error I would very much appreciate to hear about it.
Many thanks

[rbergmann (Robert Bergmann)]

@thomasneirynck I take it there is no fix for this Kibana bug yet?

[rbergmann (Robert Bergmann)]

@thomasneirynck - is there any solution for Kibana visualisations?

[rbergmann (Robert Bergmann)]

@thomasneirynck - we are looking at getting your professional support, have requested pricing information.

Just loading 40 days of indexes in Kibana Discover throws this not very useful error:

SearchError: unknown error
at https://fi-sain.teavaro.net/bundles/commons.bundle.js:5:974773
at processQueue (https://fi-sain.teavaro.net/built_assets/dlls/vendors.bundle.dll.js:499:199747)
at https://fi-sain.teavaro.net/built_assets/dlls/vendors.bundle.dll.js:499:200710
at Scope.$digest (https://fi-sain.teavaro.net/built_assets/dlls/vendors.bundle.dll.js:499:210518)
at Scope.$apply (https://fi-sain.teavaro.net/built_assets/dlls/vendors.bundle.dll.js:499:213504)
at done (https://fi-sain.teavaro.net/built_assets/dlls/vendors.bundle.dll.js:499:133057)
at completeRequest (https://fi-sain.teavaro.net/built_assets/dlls/vendors.bundle.dll.js:499:136850)
at XMLHttpRequest.requestError (https://fi-sain.teavaro.net/built_assets/dlls/vendors.bundle.dll.js:499:135713)

Any help here would be much appreciated.