# Comparing values from 2 ML Jobs in watcher
**URL:** https://discuss.elastic.co/t/comparing-values-from-2-ml-jobs-in-watcher/118882
**Category:** Elasticsearch
**Tags:** elastic-stack-machine-learning
**Created:** [February 7, 2018, 4:34pm UTC](https://discuss.elastic.co/t/comparing-values-from-2-ml-jobs-in-watcher/118882 "2018-02-07T16:34:10Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@mgarg123](https://discuss.elastic.co/u/mgarg123)
#### Post date: [February 7, 2018, 4:34pm UTC](https://discuss.elastic.co/t/comparing-values-from-2-ml-jobs-in-watcher/118882/1 "2018-02-07T16:34:10Z")
Hi,
I have 2 different ML jobs.
1. Identifies the anomaly in the users unusual login time. IE if users regularly logs in between 11:00 AM - 6:00 PM and one day he/she logs in @ 12:00 AM. this will be detected as a anomaly.
2. Detect longitude-latitude usually used by a person, if a person usually logs in fro 11.222, -11.33 (example) and suddenly logs in from 13.22, -34,55. this will be detected as anomaly. This 2 job independently works fine.
I need to write a watcher over these 2 job which determined unusual user logins and geo location at the same. IE user logged in at a unusual time and Geo location.
In the chained watcher I was able to identify both individually. How should I compare the time of the user and username from both the results and send an alert if it happens.
IE comparing the values from the 2 different job, and if then match -\> send alert. Which me comparing one array job value with other job values.
Thank for all your help
---
### Author:  [@richcollier](https://discuss.elastic.co/u/richcollier)
#### Post date: [February 7, 2018, 7:11pm UTC](https://discuss.elastic.co/t/comparing-values-from-2-ml-jobs-in-watcher/118882/2 "2018-02-07T19:11:22Z")
I don't think this is really an ML-specific question - it just can be boiled down to how does one compare lists from two separate queries from the different chained inputs, if those queries each return an array.
I sort of simulated this by running a chained input watch against 2 identical jobs (with different names) that of course, both return the same entity as anomalous.
My compare condition, however, had to "hardcode" the first entry of the results array:
```auto
"condition": {
"compare": {
"ctx.payload.first.hits.hits.0._source.partition_field_value": {
"eq": "{{ctx.payload.second.hits.hits.0._source.partition_field_value}}"
}
}
},
```
where `partition_field_value` is the field that is used to "split" the analysis on in my particular case.
The result, of course is:
```auto
"condition": {
"type": "compare",
"status": "success",
"met": true,
"compare": {
"resolved_values": {
"ctx.payload.second.hits.hits.0._source.partition_field_value": "AAL",
"ctx.payload.first.hits.hits.0._source.partition_field_value": "AAL"
}
}
}
```
But this obviously doesn't take into account if there are more than one "hits" on the results.
Hey @spinscale - is it possible to either use `array_compare` or mustache syntax to compare the array of `hits`? I think the trick in this case would be that you cannot guarantee that a particular entity is in the same index of the results array. So "AAL" might be index 0 of the `hits` array for the `first` chain, but may be in some other index location for the `second` input chain.
---
### Author:  [@mgarg123](https://discuss.elastic.co/u/mgarg123)
#### Post date: [February 7, 2018, 7:14pm UTC](https://discuss.elastic.co/t/comparing-values-from-2-ml-jobs-in-watcher/118882/3 "2018-02-07T19:14:49Z")
That's exactly the issue I am facing how can i use either java-script, painless to iterate through the 2 arrays and compare. ?
---
### Author:  [@richcollier](https://discuss.elastic.co/u/richcollier)
#### Post date: [February 7, 2018, 10:02pm UTC](https://discuss.elastic.co/t/comparing-values-from-2-ml-jobs-in-watcher/118882/4 "2018-02-07T22:02:05Z")
Ok - after a little research, this could be done in probably two ways
Method 1 - a script on the `condition` to see if there's an intersection of results from both queries
```auto
"condition": {
"script": "def second_results = ctx.payload.second.hits.hits.stream().map(hit->hit._source.partition_field_value).collect(Collectors.toList()); return ctx.payload.first.hits.hits.stream().map(hit -> hit._source.partition_field_value).filter(p->second_results.contains(p)).collect(Collectors.toList()).size() > 0;"
},
"actions": {
"log": {
"logging": {
"text": "{{ctx.payload}}"
}
}
}
```
two things to note - `first` and `second` are the names of my two chained input queries. Essentially, the `condition` script takes the anomalies for the second query, and puts them in a map/list called `second_results`. Then do the same to the first query's results, but then test is to see if there's any intersection of items from the `second_results` list (test to see if the list of matches is bigger than 0). Secondly, note again that in my specific example, it is the `partition_field_value` that contains the name of the entities that I'm interested in.
In my little test, my first query returned 3 entities:
`AAL ACA AWE`
and the second query returned 2 entities:
`ACA AAL`
and my watch returns the expected intersection:
```auto
"condition": {
"type": "script",
"status": "success",
"met": true
},
"actions": [
{
"id": "log",
"type": "logging",
"status": "success",
"logging": {
"logged_text": "{_value=[AAL, ACA]}"
}
}
]
},
```
Method 2 - a smarter filtered terms query for the second query
You could also follow the model shown in this example:
>
Where the second query does a `must` and a `terms` filter that passes all of the items from the first query as itms that must exist in the second query. It uses mustache syntax to iterate through all instances of, in this case, process names
```auto
"terms": {
"process_host": [
"{{#ctx.payload.started_processes.aggregations.process_hosts.buckets}}{{key}}",
"{{/ctx.payload.started_processes.aggregations.process_hosts.buckets}}"
]
}
```
Hope that gives you some ideas
---
### Author:  [@mgarg123](https://discuss.elastic.co/u/mgarg123)
#### Post date: [February 7, 2018, 10:10pm UTC](https://discuss.elastic.co/t/comparing-values-from-2-ml-jobs-in-watcher/118882/5 "2018-02-07T22:10:05Z")
Thanks Richcollier, this will definitely help.
---
### Author:  [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [February 7, 2018, 10:40pm UTC](https://discuss.elastic.co/t/comparing-values-from-2-ml-jobs-in-watcher/118882/6 "2018-02-07T22:40:04Z")
since Elasticsearch 6.2 you can also have transformations between two chain inputs to simplify handling of such things.
[https://www.elastic.co/guide/en/x-pack/6.2/input-chain.html](https://www.elastic.co/guide/en/x-pack/6.2/input-chain.html)
@richcollier as you already did, the correct way is to go with a scripted painless condition instead of array compare condition
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 7, 2018, 10:40pm UTC](https://discuss.elastic.co/t/comparing-values-from-2-ml-jobs-in-watcher/118882/7 "2018-03-07T22:40:23Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.