Painless script for date to minutes calculation

Hi All,

We are ingesting data using logstash http_poller, there is a date field which need to be subtracted with a default value (1970.01.01 00:00:00)

eg: result.timeworked : 1970.01.01 00:00:40

this field need to be subtracted with the default value(1970.01.01 00:00:00) , get the result as 40.

How do i achieve this? any advice on this please.

Thanks
Gautham

Hi

Have a look here:

https://www.elastic.co/guide/en/elasticsearch/painless/master/painless-datetime.html#_datetime_difference_elapsed_time

There's an example you could use for that:

ZonedDateTime zdt1 = ZonedDateTime.of(1983, 10, 13, 22, 15, 30, 11000000, ZoneId.of('Z'));
ZonedDateTime zdt2 = ZonedDateTime.of(1983, 10, 13, 22, 15, 35, 0, ZoneId.of('Z'));
long differenceInMillis = ChronoUnit.MILLIS.between(zdt1, zdt2);

this is returning milliseconds, but it shouldn't be too difficult to calculate seconds for your case
Best,
Matthias

Thanks for the response @matw , just wanted to understand the way to add the default field using logstash, is there a way to do that.
I dont have any field that contains the value 1970.01.01 00:00:00.
FIrst i need a way to add this value into elasticsearch, how do i do that, the rest api which i'm using to ingest data doesnot have this value in it.

Thanks
Gautham

You just need the seconds for this field, right? so why don't you use a numeric field in ES to store the value?
https://www.elastic.co/guide/en/elasticsearch/reference/current/number.html
Best,
Matthias

@matw i actually need to subtract two fields and get the time difference .

Field One i have it in elasticsearch, lets assume that field as "time_worked" and this field has a date value 1970.01.01 00:00:40

time_worked : 1970.01.01 00:00:40

Field Two i dont have it in elasticsearch and value of this field is a static date field, will have a value 1970.01.01 00:00:00

to use a painless script and subtract these two values i need to add the FIeld Two into elasticsearch. How do i do that?

Any advice please.

Thanks
Gautham

Since field two is a static field, so it has always the same value, right?

Yeah you are rite. It will always have the same value and it will be a date field.

If it doesn't change, why no calculating the value in log stash and storing it in a numeric field in Elasticsearch? this is the best option in this case. You can do this with a ruby filter in Logstash, here's an example that's a similar use case:


Best,
Matthias

@matw To go as per the example mentioned i dont have a second time as a field, this is a static value which i need to insert it, can i directly subtract the first date with the static date and year
StartDate - 01-01-1970 00:00:00

I dont think so this will work.

Thanks
Gautham

Yes, but you can construct a static date in the ruby script, there are several ways to do this
https://ruby-doc.org/stdlib-2.5.0/libdoc/time/rdoc/Time.html
and then use it for the subtraction
Best,
Matthias

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.