Use difference between two numeric values as query criteria in kibana

[markus (markus mayer)]

Hi,
we have a logsource that we are ingesting into kibana using logstash fileinput.

The logsource contains amon other things two numerical values. The fields are indexed as numerical values. What I'd like to do is find all events where the difference between the two numerical values is greater than X

Event-ID Foo Bar
1 10 4
2 20 18
3 30 20

So Query Foo>Bar+5 should return Event 1 and 3.

Is it possible to something like that directly in kibana?
If yes how is the syntax supposed to work.
If no should I add a third value to the venets in logstash? Do I have to do that with a custom ruby filter or are there easier options?

Bye,
Markus

[stephenb (Stephen Brown)]

@markus

Is it possible to something like that directly in kibana?

What Version? This is important

Is it possible to something like that directly in kibana?
If yes how is the syntax supposed to work.
Yes depending on the version?
What Kinda of Visualization like a Table?
Lens can do this with Custom Formula

If no should I add a third value to the venets in logstash? Do I have to do that with a custom ruby filter or are there easier options?
You can add in logstash, or you could use a runtime field

[markus (markus mayer)]

our ELK stack is on version 7.17.
Regarding what visualization. I would be glad if I could just do it ad-hoc in discover but I of course I could also setup a classic tabel or a lens table if thats possible in V7 already.

[stephenb (Stephen Brown)]

Hmmm not great but you can add a runtime field to the index pattern

code

long bar_plus_5 = doc['bar'].value + 5;
long foo = doc['foo'].value;
boolean result = false;
if (foo > bar_plus_5) {result = true;}
emit(result);

Note the result does not show up in preview...

[markus (markus mayer)]

Thx a lot for the detailed example! The runtime field works like a treat. Just out of curiosity. What other/improved ways would there be if we upgrade to V8?

Another question: While playing around with the painless script I noticed that sometimes in my kibana searches it was using an old version of my fields script. This happened even after clicking on the "refresh" button in kibana. Is the script (or its results) somehow cached? I had to change the timestamp/index mapping back and forth and then it seemed to refresh the script...

[stephenb (Stephen Brown)]

@markus

Just out of curiosity. What other/improved ways would there be if we upgrade to V8?

Well for Discover the is the new ES|QL piped query language.

For the other question, I'm not sure I'm following, but yes, the data view is cached in discover. So you have to do a shift reload on the browser to get the new data view typically? Not sure if that's really what you're asking or not.

Shouldn't have to fiddle with the time stamp. I'm not sure what you're talking about.

[markus (markus mayer)]

Thx for the pointer to the new QL piped query language feature. I'll look into this.

The second question was exactly regarding the caching/reload side of things. That got me a bit confused in the beginning but it works perfectly now.

Bye,
Markus