Scripted Fields Date Math


#1

I'd like to perform date math in a scripted field. The only two languages available are Painless and Expressions.

Want to calculate the duration between a date field and current date/time.

Here is what I have, it doesn't work and gives a compilation error.

LocalDateTime.now().value- doc['@timestamp'].value

Can someone suggest a workaround for this?

Thank you


(Brandon Kobel) #2

Painless intentionally doesn't expose .now() style methods per Groovy to Painless; get current date time

You can get the milliseconds of the document's @timestamp field using the following doc['@timestamp'].value.getMillis()


#3

Just to be clear, that will give me the number of miliseconds passed since the timestamp?


(Brandon Kobel) #4

No, it won't. doc['@timestamp'].value.getMillis() gives you the absolute number of milliseconds that the document's @timestamp field is past the epoch. Painless intentionally doesn't exist .now() style methods and recommends alternate approaches since the clocks on the various Elasticsearch nodes are potentially out of sync and occur during various times, so this likely isn't what you want to do.


#5

So is there any way of calculating how much time has passed since a timestamp?


(Brandon Kobel) #6

@kb2295 if you were using ES directly, you could pass the current datetime as a parameter to the script so we had a consistent thing to compare against, but this isn't really feasible in the context of Kibana since we don't support that.

Would you mind elaborating on the end-result that you're trying to achieve in the context of Kibana?


#7

I have a field in each doc called "lastUpdated" this is a time field that captures the time the specified doc was updated. I need to see how long it's been in real time since the doc was updated.


(Brandon Kobel) #8

Are you trying to create a chart/graph using this information, or just display it in a tabular format?


#9

I'm trying to make a chart from it.

A chart which would tell me X documents were last updated less than 30 minutes ago, Y documents were updates less than 60 mins ago...


(Brandon Kobel) #10

You can do this using the Filters aggregation like the following:


#11

So when I replace @timestamp with the name of the field, "lastUpdatedBy" it doesn't work?


(Brandon Kobel) #12

If it's a number type, no it won't. It has to be a date if you want to do date math on it.


#13

It is a Date type


(Brandon Kobel) #14

Then you'll have to use lastUpdatedBy:[now-30m TO now] and lastUpdatedBy:[now-60m TO now-30m]


#15

It works now. Thank you!

Another question. Is there documentation on the syntax?
I'm trying to figure our how to say "older than 60m" without specifying an upper bound


(Brandon Kobel) #16

Awesome, there are docs for it, it's the Lucene Query Syntax