Painless script for date to minutes calculation

Hi All,

We are ingesting data using logstash http_poller, there is a date field which need to be subtracted with a default value (1970.01.01 00:00:00)

eg: result.timeworked : 1970.01.01 00:00:40

this field need to be subtracted with the default value(1970.01.01 00:00:00) , get the result as 40.

How do i achieve this? any advice on this please.

Thanks
Gautham

Hi

Have a look here:

https://www.elastic.co/guide/en/elasticsearch/painless/master/painless-datetime.html#_datetime_difference_elapsed_time

There's an example you could use for that:

ZonedDateTime zdt1 = ZonedDateTime.of(1983, 10, 13, 22, 15, 30, 11000000, ZoneId.of('Z'));
ZonedDateTime zdt2 = ZonedDateTime.of(1983, 10, 13, 22, 15, 35, 0, ZoneId.of('Z'));
long differenceInMillis = ChronoUnit.MILLIS.between(zdt1, zdt2);

this is returning milliseconds, but it shouldn't be too difficult to calculate seconds for your case
Best,
Matthias

Thanks for the response @matw , just wanted to understand the way to add the default field using logstash, is there a way to do that.
I dont have any field that contains the value 1970.01.01 00:00:00.
FIrst i need a way to add this value into elasticsearch, how do i do that, the rest api which i'm using to ingest data doesnot have this value in it.

Thanks
Gautham

You just need the seconds for this field, right? so why don't you use a numeric field in ES to store the value?
https://www.elastic.co/guide/en/elasticsearch/reference/current/number.html
Best,
Matthias

@matw i actually need to subtract two fields and get the time difference .

Field One i have it in elasticsearch, lets assume that field as "time_worked" and this field has a date value 1970.01.01 00:00:40

time_worked : 1970.01.01 00:00:40

Field Two i dont have it in elasticsearch and value of this field is a static date field, will have a value 1970.01.01 00:00:00

to use a painless script and subtract these two values i need to add the FIeld Two into elasticsearch. How do i do that?

Any advice please.

Thanks
Gautham

Since field two is a static field, so it has always the same value, right?

Yeah you are rite. It will always have the same value and it will be a date field.

If it doesn't change, why no calculating the value in log stash and storing it in a numeric field in Elasticsearch? this is the best option in this case. You can do this with a ruby filter in Logstash, here's an example that's a similar use case:


Best,
Matthias