# After upgrade to Elastic 6.7, watcher execution fails to parse ctx.trigger.triggered\_time in watcher query **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-alerting\ **Created:** [April 23, 2019, 6:30pm UTC](https://discuss.elastic.co/t/after-upgrade-to-elastic-6-7-watcher-execution-fails-to-parse-ctx-trigger-triggered-time-in-watcher-query/178094 "2019-04-23T18:30:38Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![naziyakhan](https://avatars.discourse-cdn.com/v4/letter/n/59ef9b/32.png) [@naziyakhan](https://discuss.elastic.co/u/naziyakhan)\ **Post date:** [April 23, 2019, 6:30pm UTC](https://discuss.elastic.co/t/after-upgrade-to-elastic-6-7-watcher-execution-fails-to-parse-ctx-trigger-triggered-time-in-watcher-query/178094/1 "2019-04-23T18:30:38Z")
We have a bunch of Watchers where the Query includes time calculation using ctx.trigger.triggered\_time in the following manner: > "filter": [ - ``` { ``` - ``` "range": { ``` - ``` "@timestamp": { ``` - ``` "gte": "{{ctx.trigger.triggered_time}}||-2h", ``` - ``` "lte": "{{ctx.trigger.triggered_time}}" ``` - ``` } ``` - ``` } ``` - ``` }, ...] ``` The watchers worked fine until we upgraded Elastic to 6.7.0 Now, we have Elastic logs full of following exceptions: > org.elasticsearch.transport.RemoteTransportException: [10.3.214.4-csos-logs][10.3.214.4:9300][indices:data/read/search[phase/query]] > Caused by: org.elasticsearch.ElasticsearchParseException: failed to parse date field [{{ctx.trigger.triggered\_time}}] with format [strict\_date\_optional\_time||epoch\_millis] > at org.elasticsearch.common.joda.JodaDateMathParser.parseDateTime(JodaDateMathParser.java:215) ~[elasticsearch-6.7.0.jar:6.7.0] Is this related to the following: [Script condition | Elasticsearch Guide [6.7] | Elastic](https://www.elastic.co/guide/en/elastic-stack-overview/6.7/condition-script.html#accessing-watch-payload)? Is the {{ctx.trigger.triggered\_time}} now a string and must be transformed before its usage in Query and if yes, how can it be transformed? Any help or hint appreciated. Thanks, Naziya ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 23, 2019, 6:49pm UTC](https://discuss.elastic.co/t/after-upgrade-to-elastic-6-7-watcher-execution-fails-to-parse-ctx-trigger-triggered-time-in-watcher-query/178094/2 "2019-04-23T18:49:37Z")
Hey, can you share the full exception? I tried to reproduce this, but was not able to on 6.7.0 - I tried the following ```auto PUT foo/bar/1 { "date": "2019-04-23T12:34:56.789Z" } GET foo/_search { "query": { "range": { "date": { "gte": "2019-12-12T12:34:56.789Z||-2h", "lte": "2019-12-12T12:34:56.789Z||+2h" } } } } POST _xpack/watcher/watch/_execute { "watch": { "trigger": { "schedule": { "interval": "10h" } }, "input": { "search": { "request": { "indices": [ "foo" ], "body": { "query": { "range": { "date": { "gte": "{{ctx.trigger.triggered_time}}||-24h" } } } } } } }, "actions": { "logme": { "logging": { "text": "{{ctx}}" } } } } } ``` Also can you share the output of the execute watch api for one of those failing watches, please? Thanks a lot! --Alex ---
**Author:** ![naziyakhan](https://avatars.discourse-cdn.com/v4/letter/n/59ef9b/32.png) [@naziyakhan](https://discuss.elastic.co/u/naziyakhan)\ **Post date:** [April 23, 2019, 7:06pm UTC](https://discuss.elastic.co/t/after-upgrade-to-elastic-6-7-watcher-execution-fails-to-parse-ctx-trigger-triggered-time-in-watcher-query/178094/3 "2019-04-23T19:06:15Z")
Thanks for response Alexander. Heres the full exception and it will take sometime for me to get the watcher response, deploying the setup right now... But I should have it in the next couple of hours: > [2019-04-17T20:45:22,131][DEBUG][o.e.a.s.TransportSearchAction] [10.3.214.26-csos-logs] [172.20.105.6-logs-2019.16][3], node[Faa3d0SgT3-IO43lbakRMg], [R], s[STARTED], a[id=DsSm7Jw6TvW2vB82NY5fzg]: Failed to execute [SearchRequest{searchType=QUERY\_THEN\_FETCH, indices=[\<\*-logs-{now/w{yyyy.ww}}\>], indicesOptions=IndicesOptions[ignore\_unavailable=false, allow\_no\_indices=true, expand\_wildcards\_open=true, expand\_wildcards\_closed=false, allow\_aliases\_to\_multiple\_indices=true, forbid\_closed\_indices=true, ignore\_aliases=false, ignore\_throttled=true], types=, routing='null', preference='null', requestCache=null, scroll=null, maxConcurrentShardRequests=10, batchedReduceSize=512, preFilterShardSize=128, allowPartialSearchResults=true, localClusterAlias=null, getOrCreateAbsoluteStartMillis=-1, source={"query":{"bool":{"filter":[{"range":{"@timestamp":{"from":"{{ctx.trigger.triggered\_time}}||-2h","to":"{{ctx.trigger.triggered\_time}}||","include\_lower":true,"include\_upper":true,"boost":1.0}}},{"match\_phrase":{"message":{"query":"Event","slop":0,"zero\_terms\_query":"NONE","boost":1.0}}},{"match\_phrase":{"message":{"query":"has entered maintenance mode","slop":0,"zero\_terms\_query":"NONE","boost":1.0}}}],"must\_not":[{"exists":{"field":"tagged","boost":1.0}},{"match\_phrase":{"message":{"query":"[Unknown user]","slop":0,"zero\_terms\_query":"NONE","boost":1.0}}}],"adjust\_pure\_negative":true,"boost":1.0}}}}] > org.elasticsearch.transport.RemoteTransportException: [10.3.214.4-csos-logs][10.3.214.4:9300][indices:data/read/search[phase/query]] > Caused by: org.elasticsearch.ElasticsearchParseException: failed to parse date field [{{ctx.trigger.triggered\_time}}] with format [strict\_date\_optional\_time||epoch\_millis] > at org.elasticsearch.common.joda.JodaDateMathParser.parseDateTime(JodaDateMathParser.java:215) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.common.joda.JodaDateMathParser.parse(JodaDateMathParser.java:69) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.parseToMilliseconds(DateFieldMapper.java:316) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.isFieldWithinQuery(DateFieldMapper.java:329) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.query.RangeQueryBuilder.getRelation(RangeQueryBuilder.java:459) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.query.RangeQueryBuilder.doRewrite(RangeQueryBuilder.java:476) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.query.AbstractQueryBuilder.rewrite(AbstractQueryBuilder.java:284) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.query.BoolQueryBuilder.rewriteClauses(BoolQueryBuilder.java:485) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.query.BoolQueryBuilder.doRewrite(BoolQueryBuilder.java:452) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.query.AbstractQueryBuilder.rewrite(AbstractQueryBuilder.java:284) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.builder.SearchSourceBuilder.rewrite(SearchSourceBuilder.java:949) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.builder.SearchSourceBuilder.rewrite(SearchSourceBuilder.java:80) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:68) ~[elasticsearch-6.7.0.jar:6.7.0] ---
**Author:** ![naziyakhan](https://avatars.discourse-cdn.com/v4/letter/n/59ef9b/32.png) [@naziyakhan](https://discuss.elastic.co/u/naziyakhan)\ **Post date:** [April 23, 2019, 7:06pm UTC](https://discuss.elastic.co/t/after-upgrade-to-elastic-6-7-watcher-execution-fails-to-parse-ctx-trigger-triggered-time-in-watcher-query/178094/4 "2019-04-23T19:06:33Z")
> Second half of the exception: > at org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:51) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.internal.ShardSearchLocalRequest$RequestRewritable.rewrite(ShardSearchLocalRequest.java:307) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.internal.ShardSearchLocalRequest$RequestRewritable.rewrite(ShardSearchLocalRequest.java:297) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:68) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService.createSearchContext(SearchService.java:671) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService.createSearchContext(SearchService.java:651) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService.createContext(SearchService.java:614) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:595) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:386) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService.access$100(SearchService.java:125) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:358) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:354) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService$4.doRun(SearchService.java:1085) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) [elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:751) [elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.7.0.jar:6.7.0] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0\_161] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0\_161] > at java.lang.Thread.run(Thread.java:748) [?:1.8.0\_161] > Caused by: java.lang.IllegalArgumentException: Parse failure at index [0] of [{{ctx.trigger.triggered\_time}}] > at org.elasticsearch.common.joda.JodaDateMathParser.parseDateTime(JodaDateMathParser.java:208) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.common.joda.JodaDateMathParser.parse(JodaDateMathParser.java:69) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.parseToMilliseconds(DateFieldMapper.java:316) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.isFieldWithinQuery(DateFieldMapper.java:329) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.query.RangeQueryBuilder.getRelation(RangeQueryBuilder.java:459) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.query.RangeQueryBuilder.doRewrite(RangeQueryBuilder.java:476) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.query.AbstractQueryBuilder.rewrite(AbstractQueryBuilder.java:284) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.query.BoolQueryBuilder.rewriteClauses(BoolQueryBuilder.java:485) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.query.BoolQueryBuilder.doRewrite(BoolQueryBuilder.java:452) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.query.AbstractQueryBuilder.rewrite(AbstractQueryBuilder.java:284) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.builder.SearchSourceBuilder.rewrite(SearchSourceBuilder.java:949) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.builder.SearchSourceBuilder.rewrite(SearchSourceBuilder.java:80) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:68) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:51) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.internal.ShardSearchLocalRequest$RequestRewritable.rewrite(ShardSearchLocalRequest.java:307) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.internal.ShardSearchLocalRequest$RequestRewritable.rewrite(ShardSearchLocalRequest.java:297) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:68) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService.createSearchContext(SearchService.java:671) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService.createSearchContext(SearchService.java:651) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService.createContext(SearchService.java:614) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:595) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:386) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService.access$100(SearchService.java:125) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:358) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:354) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.search.SearchService$4.doRun(SearchService.java:1085) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:751) ~[elasticsearch-6.7.0.jar:6.7.0] > at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-6.7.0.jar:6.7.0] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0\_161] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0\_161] > at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0\_161] ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 24, 2019, 8:54pm UTC](https://discuss.elastic.co/t/after-upgrade-to-elastic-6-7-watcher-execution-fails-to-parse-ctx-trigger-triggered-time-in-watcher-query/178094/5 "2019-04-24T20:54:30Z")
can you share the full watch? And also the version you upgraded from where this still worked? --Alex ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 22, 2019, 8:54pm UTC](https://discuss.elastic.co/t/after-upgrade-to-elastic-6-7-watcher-execution-fails-to-parse-ctx-trigger-triggered-time-in-watcher-query/178094/6 "2019-05-22T20:54:30Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.