Substract values before applying aggregation


(Guim RH) #1

Hello,
we have an index with two fields (amount others) one is the scheduled time of one job, and the second one is the actual execution time, this second value is used as @timestamp.
We'd like to have the delay between scheduling and execution for each individual jobs before applying any agregation.

So far I got this:
.es(index=myindex-*,metric=avg:@timestamp).subtract(.es(index=myindex-*,metric=avg:@scheduledTime))
but I'm not sure it is the solution I want.
This is first calculating the average value of those fields (wich are timestamps) and then subtracting the results.
I would like to first substract the values of those fields for each document (so I get the delay of each particular job), and then apply the aggregation I like (max, avg...).
something like that:
.es(index=myindex-*,q="(term:@timestamp)-(term:@scheduledTime)")

Maybe the solution here is to obtain the delay on the source, in my logstash. This solution would also imply we'd had another field (the delay itself) on our index.

Thank you for your help.


(Guim RH) #2

Well,
I got an alternative, but I don't know if it's the most efficient.
You can define new fields with scripting: https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-scripting.html

with this you set the rule for a new field, in my case:
doc['@timestamp'].value-doc['@scheduledTime'].value


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.