# Ability to search ES based on a result of previous search in a watcher **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-alerting\ **Created:** [March 23, 2020, 4:34pm UTC](https://discuss.elastic.co/t/ability-to-search-es-based-on-a-result-of-previous-search-in-a-watcher/224693 "2020-03-23T16:34:41Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Jasonespo](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@Jasonespo](https://discuss.elastic.co/u/Jasonespo)\ **Post date:** [March 23, 2020, 4:34pm UTC](https://discuss.elastic.co/t/ability-to-search-es-based-on-a-result-of-previous-search-in-a-watcher/224693/1 "2020-03-23T16:34:41Z")
Hi I was wondering if it was possible (in a watcher) to do the following: - Search an index for a keyword e.g document must contain "tag": "laptop" - In that document it contains another field called "identifier": "091029" - Search again for all documents with the same "identifier": "091029" within the past X time period - Display the result in json Any ideas? Thanks ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [March 24, 2020, 4:42pm UTC](https://discuss.elastic.co/t/ability-to-search-es-based-on-a-result-of-previous-search-in-a-watcher/224693/2 "2020-03-24T16:42:09Z")
Hey Jason, can you explain the connection between those two queries? Right now for me these two look like independent queries, so I guess I am missing something. For running two queries as part of a watch input, check out the [chain input](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/input-chain.html) Hope this helps! --Alex ---
**Author:** ![Jasonespo](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@Jasonespo](https://discuss.elastic.co/u/Jasonespo)\ **Post date:** [March 25, 2020, 10:45am UTC](https://discuss.elastic.co/t/ability-to-search-es-based-on-a-result-of-previous-search-in-a-watcher/224693/3 "2020-03-25T10:45:29Z")
@spinscale - Looks like a potential so far, thank you. Not 100% sure it operates how i want it to though.. Let me try and explain a bit further... The first search is to alert/execute if any logs come in with the tag "laptop" (note this will only execute if the tag is "laptop". The document will have another field called identifier which I will need to set as a variable for the next search. ``` { "trigger": { "schedule": { "interval": "5m" } }, "input": { "chain": { "indices": [ "tid*" ], "inputs": [ { "first": { "simple": { "query": { "bool": { "filter": [ { "range": { "@timestamp": { "gte": "now-{{ctx.metadata.window_period}}" } } }, { "term": { "tag": "laptop" } } ] } } } } }, ``` After the first search returns the document I then need to do the second search on the identifier variable ``` ... "second": { "simple": { "query": { "bool": { "filter": [ { "range": { "@timestamp": { "gte": "now-{{ctx.metadata.window_period}}" } } }, { "term": { "identifier": $value } } ] } } } } }... ``` Then the json body result would be all the documents with that identifier $value in the past 2 minutes.. Do you think this is possible? And any guidance on accessing the initial document to set the variable for the second search? Thanks for the help so far! ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [March 30, 2020, 1:50pm UTC](https://discuss.elastic.co/t/ability-to-search-es-based-on-a-result-of-previous-search-in-a-watcher/224693/4 "2020-03-30T13:50:33Z")
Yes, you can do this. You can convert the data using a script transform in your chain inputs. See [https://www.elastic.co/guide/en/elasticsearch/reference/7.6/input-chain.html#\_transforming\_chained\_input\_data](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/input-chain.html#_transforming_chained_input_data) ---
**Author:** ![Jasonespo](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@Jasonespo](https://discuss.elastic.co/u/Jasonespo)\ **Post date:** [March 30, 2020, 2:41pm UTC](https://discuss.elastic.co/t/ability-to-search-es-based-on-a-result-of-previous-search-in-a-watcher/224693/5 "2020-03-30T14:41:16Z")
Thanks again @spinscale I've been looking at this a while now and can't get it to work even with transforms and chain inputs... (It probably is the solution).. Let me give you the _exact_ use case and if you could confirm one last time that it would work for this, that would be great (then i'll get out of your hair!) We want to alert on certain audit logs. These logs have a "key" field. The watcher would look for the occurence of this "key" field. (Relatively simple start, I know how to do this). After it has matched a certain key value e.g ``` "key": "alert_key" ``` However, the document that contains this field does **not** have all the relevant information that we want to produce to our users. It contains another field called "alert\_id". This unique alert\_id field exists in 5 or more documents, **however the previous key we searched for does not exist in these documents** (otherwise this would be to easy.. eh..) I need to take a different field from the initial document to be used in the second search... This would be alert\_id. ``` "alert_id": "01723213-0183291013" ``` We now after finding this "alert\_id" need to find ALL the documents with this unique key in the past X time period. I'm probably out of my depth attempting this and need to seek help from one of other members of the team.. Maybe [this](https://www.elastic.co/guide/en/elasticsearch/reference/current/transform-chain.html) is more my solution - what do you think Tried it a bit here: ``` { "trigger": { "schedule": { "interval": "2m" } }, "transform": { "chain": [ { "search": { "request": { "indices": [ "tid*" ], "body": { "size": 0, "query": { "bool": { "filter": [ { "range": { "@timestamp": { "gte": "now-{{ctx.metadata.window_period}}" } } }, { "term": { "key": "alert_key" } }, { "aggs": { "term": { "field": "alert_id" } } } ] }, "query": { "bool": { "filter": [ { "range": { "@timestamp": { "gte": "now-{{ctx.metadata.window_period}}" } } }, { //not sure how to query Kibana for the alert_id value from the previous query } ] } } } } } } } ] } } ``` Thanks again. ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [March 30, 2020, 3:16pm UTC](https://discuss.elastic.co/t/ability-to-search-es-based-on-a-result-of-previous-search-in-a-watcher/224693/6 "2020-03-30T15:16:44Z")
Check out this snippet, hope it helps ```auto DELETE first_index,second_index PUT first_index/_doc/1?refresh { "field_to_check" : "foo" } PUT second_index/_doc/1 { "key" : "foo" } PUT second_index/_doc/2?refresh { "key" : "bar" } POST _xpack/watcher/watch/_execute { "watch": { "trigger": { "schedule": { "interval": "10h" } }, "input": { "chain": { "inputs": [ { "first": { "search": { "request": { "indices": [ "first_index" ], "body": { "query": { "match_all": {} } } } } } }, { "second": { "search": { "request": { "indices": [ "second_index" ], "body": { "query": { "term": { "key": { "value": "{{ctx.payload.first.hits.hits.0._source.field_to_check}}" } } } } } } } } ] } }, "actions": { "logme": { "logging": { "text": "{{ctx}}" } } } } } ``` ---
**Author:** ![Jasonespo](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@Jasonespo](https://discuss.elastic.co/u/Jasonespo)\ **Post date:** [March 30, 2020, 3:59pm UTC](https://discuss.elastic.co/t/ability-to-search-es-based-on-a-result-of-previous-search-in-a-watcher/224693/7 "2020-03-30T15:59:57Z")
GREAT, thank you! Looks like we are getting there... This would match every document with the field named "field\_to\_check". So for us this would be "alert\_id". But we only care about matching the values of "field\_to\_check" if it matches what was in the initial query... So of the 10 documents that contain that field name... we might only need 3 of them. Would the next step be a condition that compares the alert\_ids in query 1 and query 2? Then output the offenders that match the condition? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 27, 2020, 4:14pm UTC](https://discuss.elastic.co/t/ability-to-search-es-based-on-a-result-of-previous-search-in-a-watcher/224693/8 "2020-04-27T16:14:45Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.