Painless scripts in Kibana: inline vs stored scripts

[mbelluomini]

Hi everybody.
In a (bar chart) visualization I'm successfully using in the JSON input field (advanced area of the Y-axis) the following JSON Painless script (in order to get the ratio ab2_tot_immobilizz_mater/ab1_tot_immobilizz_immater - both of them are mapped as numbers (double) in a ES index):

{"script": {"lang": "painless","source": "double score = 0.0; if (doc['ab1_tot_immobilizz_immater'].value != 0) score += doc['ab2_tot_immobilizz_mater'].value / doc['ab1_tot_immobilizz_immater'].value; return score"} }

Just to give all useful info, In the same Y-axis area settings the aggregation is the 'average' on the field ab2_tot_immobilizz_mater

Doing so the chart is correctly drawn and everything is fine.

Then I'd wish to store the script in ES, so to recall the script ID in kibana instead of write it in the visualization code. Hence, I (succesfully) executed the following statement in Dev tools:

POST /_scripts/mat_div_imm?pretty
{"script": {"lang": "painless","source": "double score = 0.0; if (doc['ab1_tot_immobilizz_immater'].value != 0) score += doc['ab2_tot_immobilizz_mater'].value / doc['ab1_tot_immobilizz_immater'].value; return score"} }

hoping that writing the following JSON in the JSON input field of the advanced setting of the Y-axis would be ok:

{"script": {"lang": "painless", "id": "mat_div_imm" }}

Unfortunately Kibana report the following error:

Visualize: [x_content_parse_exception] [1:211] [avg] failed to parse field [script]

Anyone could be have hints or be able to underline what and where I did wrong?

Thanks in advance for any help from you all.
M.

[lukas (Lukas Olson)]

Hmm, I'm not sure why this isn't working off the top of my head, but is there a reason you're not using scripted fields to accomplish this instead? You can manage scripted fields in the index pattern settings.

[mbelluomini]

Hi Lukas,
although (at least to me) the original issue I arose stays I want to thank you so much about using scripted fields.
I explored the chance to use them and I succeeded being able to create a visualization and to save it in a dashboard. Unfortunately I don't understand how to make the scripted field "persistent" in Kibana.
What I mean is opening Kibana from another browser session the scripted field disappear (is not listed anymore) and the related visualization into the dashboard doesn't work anymore too.

Thx in advance for any additional hints and new support/help,
M.

[mbelluomini]

I forgot to write down the error message I am getting from the dashboard visualization attempt:
"
There is a problem with this saved object
A field associated with this object no longer exists in the index pattern.
If you know what this error means, go ahead and fix it — otherwise click the delete button above.
"

And also I was forgetting to underline the index pattern is referring to a set of indexes. in other words, to be clearer, the index pattern is like 'index*' and the real indexes are the following seven 'index_2011', 'index_2012',.., 'index_2017'.
Trying to experiment the issue on a index pattern related to a very single index, I saw the scripted field is 'persistent' !!!

Could it be possible the issue ( I mean on multi indexes index pattern) come from a bug?

Thx so much again for any help,
M.

[lukas (Lukas Olson)]

Hmm, yes, that sounds like it could be a bug. Do you have multiple tabs open? I would try closing all tabs, creating the scripted field, then re-opening any tabs. There was a bug where if you kept Discover open it would clear out your scripted fields because it was setting popularity counters on fields.

[mbelluomini]

Hi Lukas,
You were right !!!

I did it again using just one tab for any interaction with Kibana and everything was having no issues.
I can confirm: using multiple session of Kibana (on multiple browser tabs/windows) can erase the scripted fields settings. It is very recommended to have just a single browser session open while setting up scripted fields.

So, finally, there's no (direct) bug on Kibana!
A personal suggestion: it could be really useful for end user to get this kind of warning in Kibana online documentation about scripting and scripted fields usage.

Endless thanks Lukas, you definitely rules !

So nice to meet and interact with you, my best regards,
M.

[lukas (Lukas Olson)]

I'm glad we were able to get this figured out! It actually was a bug that is fixed in 6.4+. Here's the ticket that was associated with it: https://github.com/elastic/kibana/issues/15738