# Ctx.payload.hits.hits.0.\_source.field to watcher-history index **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-alerting\ **Created:** [April 11, 2018, 3:39pm UTC](https://discuss.elastic.co/t/ctx-payload-hits-hits-0-source-field-to-watcher-history-index/127666 "2018-04-11T15:39:48Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![Alex\_Kefallonitis](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@Alex\_Kefallonitis](https://discuss.elastic.co/u/Alex_Kefallonitis)\ **Post date:** [April 11, 2018, 3:39pm UTC](https://discuss.elastic.co/t/ctx-payload-hits-hits-0-source-field-to-watcher-history-index/127666/1 "2018-04-11T15:39:48Z")
Hi all, I am trying to aggregate ctx source fields (triggered source event fields) and force watcher to write them to watcher-history index. For eg i wanna pass the ctx.payload.hits.hits.0.\_source.computer\_name of a winlogbeat as new field to the watcher-history index. Is there a way? I tried tranform script , webhook and update\_query but none of the seem to work as expected. I really could use an example . Thanks in advanced for any help ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 11, 2018, 3:55pm UTC](https://discuss.elastic.co/t/ctx-payload-hits-hits-0-source-field-to-watcher-history-index/127666/2 "2018-04-11T15:55:53Z")
the watcher history index mapping should not be fiddled with. However, all the results of a search are already written into a watch record (the name for a single entry in the history), albeit not searchable. If you need to have this searchable you should probably reindex the data in your own index. When doing that you might want to take a look at the `result.input` field which contains your search. Hope this helps. --Alex ---
**Author:** ![Alex\_Kefallonitis](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@Alex\_Kefallonitis](https://discuss.elastic.co/u/Alex_Kefallonitis)\ **Post date:** [April 11, 2018, 6:42pm UTC](https://discuss.elastic.co/t/ctx-payload-hits-hits-0-source-field-to-watcher-history-index/127666/3 "2018-04-11T18:42:54Z")
I need to write ctx.payload.hits.hits.0.\_source.field or result.input.payload.hits.hits fields (if accessible) to watcher-history index or another index if it is possible and write watches that correlates the correlated events. I understand why i shouldn't write to watcher-history index but i need to have the fields accessed and searched. Is it possible? Can a watcher action write payload fields to an index to map them? I would appreciate an example . Thanks again for your help ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 11, 2018, 6:54pm UTC](https://discuss.elastic.co/t/ctx-payload-hits-hits-0-source-field-to-watcher-history-index/127666/4 "2018-04-11T18:54:20Z")
How about using the `index` action to write to an index you maintain yourself? ---
**Author:** ![Alex\_Kefallonitis](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@Alex\_Kefallonitis](https://discuss.elastic.co/u/Alex_Kefallonitis)\ **Post date:** [April 11, 2018, 9:05pm UTC](https://discuss.elastic.co/t/ctx-payload-hits-hits-0-source-field-to-watcher-history-index/127666/5 "2018-04-11T21:05:48Z")
I will try something like that: ``` "index_payload": { "index": { "index": "correlated", "doc_type": "data" } } ``` This will populate the fields from the watch ? ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 11, 2018, 10:10pm UTC](https://discuss.elastic.co/t/ctx-payload-hits-hits-0-source-field-to-watcher-history-index/127666/6 "2018-04-11T22:10:30Z")
the current payload will be used, but you may likely want to use a script transform before the action is executed. There is even support to index into multiple documents, see [https://www.elastic.co/guide/en/x-pack/6.2/actions-index.html](https://www.elastic.co/guide/en/x-pack/6.2/actions-index.html) ---
**Author:** ![Alex\_Kefallonitis](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@Alex\_Kefallonitis](https://discuss.elastic.co/u/Alex_Kefallonitis)\ **Post date:** [April 12, 2018, 7:58am UTC](https://discuss.elastic.co/t/ctx-payload-hits-hits-0-source-field-to-watcher-history-index/127666/7 "2018-04-12T07:58:25Z")
I tried transforming a field. "index\_payload": { "transform": { "script": { "source": "['sourceHostname' : ctx.payload.hits.hits.0.\_source.computer\_name]", "lang": "painless" } }, "index": { "index": "correlated", "doc\_type": "data" } } Watch got a hit the result was the following: ``` { ``` "id": "index\_payload", "type": "index", "status": "success", "transform": { "type": "script", "status": "success", "payload": { "sourceHostname": "xxxxx" } }, "index": { "response": { "created": true, "result": "created", "id": "UGrQuGIBIQgb1Is59367", "version": 1, "type": "data", "index": "correlated" } } } And although the response is successive in the correlated index i see nothing. Note that without transform hit is indexed in the correlated. Any clues? Thanks again ---
**Author:** ![Alex\_Kefallonitis](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@Alex\_Kefallonitis](https://discuss.elastic.co/u/Alex_Kefallonitis)\ **Post date:** [April 12, 2018, 8:07am UTC](https://discuss.elastic.co/t/ctx-payload-hits-hits-0-source-field-to-watcher-history-index/127666/8 "2018-04-12T08:07:30Z")
Update: Only sourceHostname is populated in the index . Is there a way to loop with transform to extract all fields and populated the recursively? Thanks for any help ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 12, 2018, 8:15am UTC](https://discuss.elastic.co/t/ctx-payload-hits-hits-0-source-field-to-watcher-history-index/127666/9 "2018-04-12T08:15:47Z")
Do you want to have all source hostnames? ```auto def hostnames = ctx.payload.hits.hits.stream().map(hit -> hit._source.computer_name).collect(Collectors.toList()); ``` ---
**Author:** ![Alex\_Kefallonitis](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@Alex\_Kefallonitis](https://discuss.elastic.co/u/Alex_Kefallonitis)\ **Post date:** [April 12, 2018, 8:19am UTC](https://discuss.elastic.co/t/ctx-payload-hits-hits-0-source-field-to-watcher-history-index/127666/10 "2018-04-12T08:19:44Z")
Nope i want all the field from the hit to populated to the new index through a transform loop. And not add them manually. Or maybe pass someway the hits json as a whole? I am not sure which way is better. An example could help. Thanks again ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 12, 2018, 8:47am UTC](https://discuss.elastic.co/t/ctx-payload-hits-hits-0-source-field-to-watcher-history-index/127666/11 "2018-04-12T08:47:34Z")
well, then dont filter on the field name but just use the whole `_source` in the above example. We have a couple of [examples](https://github.com/elastic/examples/tree/master/Alerting), you might want to take a look at. As you are testing with out own data, I think the most important part would be to reduce your feedback loop while testing. I wrote a longish blog post on that some time ago, which I can just advice to read. See [https://www.elastic.co/blog/watching-the-watches-writing-debugging-and-testing-watches](https://www.elastic.co/blog/watching-the-watches-writing-debugging-and-testing-watches) ---
**Author:** ![Alex\_Kefallonitis](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@Alex\_Kefallonitis](https://discuss.elastic.co/u/Alex_Kefallonitis)\ **Post date:** [April 12, 2018, 8:55am UTC](https://discuss.elastic.co/t/ctx-payload-hits-hits-0-source-field-to-watcher-history-index/127666/12 "2018-04-12T08:55:35Z")
``` "index_payload": { "transform": { "script": { "source": "[ctx.payload.hits.hits.0._source]", "lang": "painless" } }, ``` OR ``` "index_payload": { "transform": { "script": { "source": "['message': ctx.payload.hits.hits.0._source]", "lang": "painless" } }, ``` Something like that you mean? Thanks for the examples i really wanted 2 lines of an example for my case ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 12, 2018, 9:02am UTC](https://discuss.elastic.co/t/ctx-payload-hits-hits-0-source-field-to-watcher-history-index/127666/13 "2018-04-12T09:02:07Z")
sorry for not being, I was referring to my example above, so you include all the hits. ---
**Author:** ![Alex\_Kefallonitis](https://avatars.discourse-cdn.com/v4/letter/a/f9ae1b/32.png) [@Alex\_Kefallonitis](https://discuss.elastic.co/u/Alex_Kefallonitis)\ **Post date:** [April 12, 2018, 9:47am UTC](https://discuss.elastic.co/t/ctx-payload-hits-hits-0-source-field-to-watcher-history-index/127666/14 "2018-04-12T09:47:11Z")
No worries thanks. One last question can i add in the same script multiple field values. For eg: "source": "['sourceHostname' : ctx.payload.hits.hits.0.\_source.computer\_name, 'destinationHostname' : ctx.payload.hits.hits.0.\_source.test\_field]" Whould it work? Thanks again ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 12, 2018, 2:22pm UTC](https://discuss.elastic.co/t/ctx-payload-hits-hits-0-source-field-to-watcher-history-index/127666/15 "2018-04-12T14:22:28Z")
Hey, yes that would work. my example above simply copies all fields. Also you could of course create black/whitelist to only include a certain set of fields, without having to specify them manually. --Alex ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 10, 2018, 2:22pm UTC](https://discuss.elastic.co/t/ctx-payload-hits-hits-0-source-field-to-watcher-history-index/127666/16 "2018-05-10T14:22:52Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.