Scripted fields if value exists

Elastic Stack Kibana
1

I'm desperately trying to use the scripted fields feature of Kibana in order to apply a minor transform on a display value. Based on everything I've read in the scripted fields guide and scripts in aggregation guide, I should be able to do something as simple as the following:

if (doc['field_name'].value > 0) { (ceil(doc['field_name'].value) / 1000) / 60 }

But when I do that, I end up with an error on every page that says "Discover: An error occurred with your request. Reset your inputs and try again."

Furthermore, if I simply transform the value:

(ceil(doc['field_name'].value) / 1000) / 60

Everything works great, except when I run a query which contains elements that don't have the doc['field_name'] in question...hence my attempt at an "if" statement.

Does anyone have a good resource for scripting these fields, or see anything wrong with my logic?

2

Default scripting language for Kibana 4 scripted fields is Lucene expressions, which has limited support for conditionals (only a ternary operator, no "if"). Alternatively, you can try using static Groovy scripts, described here: Calling groovy script from Kibana

3

Yes, I came across the ternary operator, but couldn't get that working either :(. I expected this...

(doc['field_name'].empty) ? 0 : (ceil(doc['field_name'].value) / 1000) / 60

to produce the desired result. It did not. I assume it's because the simple test for doc['field_name'] errors since the field may not exist within the resultset.

I would imagine there is an easy way to rule that out, however I can't seem to find it :frowning: .

Thank you for your response :smile:

4

Kibana scripted fields documentation says: "If a field is sparse (only some documents contain a value), documents missing the field will have a value of 0", which leads me to believe that your expression (ceil(doc['field_name'].value) / 1000) / 60 should work without a conditional. What error are you getting when you use that?

5

I get that message that says "Discover: An error occurred with your request. Reset your inputs and try again." anytime I execute a query that has a timeframe wider than the dataset. Also, the data I do have fails to load.

For example, I have 7 days worth of data containing the "field_name." If I use the Discover tab and ask it to give me the last 90 days of data, it returns 7 days worth of data with no error. If I add a scripted field to the system containing just my simple math, and ask for anything over 7 days worth of data, I get that error, and the UI spins forever with no return values.

6

Just to be sure, are you replacing "field_name" with the name of your field? I indexed a document that is missing the a value for "bytes", which I'm referencing in the same script you tried above. I haven't gotten any failures in Discover and the value of the scripted field is 0, as expected (see screenshots below), without the need for conditionals.

That's not to say that there isn't something else odd going on...

Screen Shot 2015-06-18 at 10.27.27 PM.png948×164 17.5 KB

Screen Shot 2015-06-18 at 10.27.08 PM.png924×1498 215 KB

7

Ya, absolutely :). I've verified the field exists, in fact...it does work as long as I don't hit the buffer of days.

8

This should work

(doc['filesize'].value > 0 && doc['response_time'].value > 0) (doc['filesize'].value)/(doc['response_time'].value) : 0

9

Hi

Can anyone help me in this .
i created a scripted field in kibana
doc['used memory'].value / doc['total memory'].value
but the output is not returning anything,the value is blank.

reply.png470×1136 78.2 KB

10

I have a same question. In my case only some documents contain a value. If I use script like this doc['field_name'].value) / 1000 then all empty fields will fill with 0. It leads to the worng avg value in diagramms. I've try to use script with if-then-else-condition like this ! doc['field_name'].empty ? doc['field_name'].value / 1000, but in this case I need also else condition. or?

Is there some opption to use do nothing as else-condition?
! doc['field_name'].empty ? doc['field_name'].value / 1000 : do nothing

1 Like