Painless script for date to minutes calculation

[Gauti (Gautham)]

Hi All,

We are ingesting data using logstash http_poller, there is a date field which need to be subtracted with a default value (1970.01.01 00:00:00)

eg: result.timeworked : 1970.01.01 00:00:40

this field need to be subtracted with the default value(1970.01.01 00:00:00) , get the result as 40.

How do i achieve this? any advice on this please.

Thanks
Gautham

[matw (Matthias Wilhelm)]

Hi

Have a look here:

https://www.elastic.co/guide/en/elasticsearch/painless/master/painless-datetime.html#_datetime_difference_elapsed_time

There's an example you could use for that:

ZonedDateTime zdt1 = ZonedDateTime.of(1983, 10, 13, 22, 15, 30, 11000000, ZoneId.of('Z'));
ZonedDateTime zdt2 = ZonedDateTime.of(1983, 10, 13, 22, 15, 35, 0, ZoneId.of('Z'));
long differenceInMillis = ChronoUnit.MILLIS.between(zdt1, zdt2);

this is returning milliseconds, but it shouldn't be too difficult to calculate seconds for your case
Best,
Matthias

[Gauti (Gautham)]

Thanks for the response @matw , just wanted to understand the way to add the default field using logstash, is there a way to do that.
I dont have any field that contains the value 1970.01.01 00:00:00.
FIrst i need a way to add this value into elasticsearch, how do i do that, the rest api which i'm using to ingest data doesnot have this value in it.

Thanks
Gautham

[matw (Matthias Wilhelm)]

You just need the seconds for this field, right? so why don't you use a numeric field in ES to store the value?
https://www.elastic.co/guide/en/elasticsearch/reference/current/number.html
Best,
Matthias

[Gauti (Gautham)]

@matw i actually need to subtract two fields and get the time difference .

Field One i have it in elasticsearch, lets assume that field as "time_worked" and this field has a date value 1970.01.01 00:00:40

time_worked : 1970.01.01 00:00:40

Field Two i dont have it in elasticsearch and value of this field is a static date field, will have a value 1970.01.01 00:00:00

to use a painless script and subtract these two values i need to add the FIeld Two into elasticsearch. How do i do that?

Any advice please.

Thanks
Gautham

[matw (Matthias Wilhelm)]

Since field two is a static field, so it has always the same value, right?

[Gauti (Gautham)]

Yeah you are rite. It will always have the same value and it will be a date field.

[matw (Matthias Wilhelm)]

If it doesn't change, why no calculating the value in log stash and storing it in a numeric field in Elasticsearch? this is the best option in this case. You can do this with a ruby filter in Logstash, here's an example that's a similar use case:

Get Difference between two date field with ruby filter

Hi All I have been trying the ruby code to get the difference between 2 dates Start Date : 2017-06-13T01:17:07.000Z End Date : 2017-07-13T01:17:10.000Z I need difference between these 2 dates. I have dried this but i get ruby exception error ruby { init => "require 'time'" code => " start = Time.iso8601(event['Start Date'].to_s).to_i; end = Time.iso8601(event['End Date'].to_s).to_i; event['timediff'] = start - end; " add_tag => [ "calculat…

Best,
Matthias

[Gauti (Gautham)]

@matw To go as per the example mentioned i dont have a second time as a field, this is a static value which i need to insert it, can i directly subtract the first date with the static date and year
StartDate - 01-01-1970 00:00:00

I dont think so this will work.

Thanks
Gautham

[matw (Matthias Wilhelm)]

Yes, but you can construct a static date in the ruby script, there are several ways to do this
https://ruby-doc.org/stdlib-2.5.0/libdoc/time/rdoc/Time.html
and then use it for the subtraction
Best,
Matthias