Can't insert date into watch JSON via templating and script

I created a new watch for altering the output of a machine learning job. The watch JSON looks like below.

{
"trigger": {
"schedule": {
"interval": "1m"
}
},
"input": {
"http": {
"request": {
"scheme": "http",
"host": "localhost",
"port": 9200,
"method": "get",
"path": "/_xpack/ml/anomaly_detectors/total-requests/results/buckets",
"params": {},
"headers": {},
"body": "{ "anomaly_score": 75, "start": "{{ctx.payload.time}}" }"
}
}
},
"condition": {
"always": {}
},
"actions": {
"notify-slack": {
"throttle_period_in_millis": 300000,
"slack": {
"message": {
"to": [
"#test-alerts"
],
"text": "New anomaly detected with score {{ctx.payload.score}}."
}
}
}
},
"transform": {
"script": {
"source": "return [ 'time' : Instant.ofEpochMilli(ctx.execution_time.getMillis()).minus(1, ChronoUnit.HOURS)]",
"lang": "painless"
}
}
}

However when the watch is triggered, it raises the following error:

{
"result": {
"execution_time": "2019-03-21T16:05:40.768Z",
"execution_duration": 6,
"input": {
"type": "http",
"status": "success",
"payload": {
"_headers": {
"content-type": [
"application/json; charset=UTF-8"
]
},
"error": {
"root_cause": [
{
"type": "parse_exception",
"reason": "failed to parse date field with format [strict_date_optional_time||epoch_millis]"
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": ".ml-anomalies-shared",
"node": "Hmt2nz5fQcSeBgdNmcEoVA",
"reason": {
"type": "parse_exception",
"reason": "failed to parse date field with format [strict_date_optional_time||epoch_millis]",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "Parse failure at index [0] of "
}
}
}
]
},
"_status_code": 400,
"status": 400
},
"http": {
"request": {
"host": "localhost",
"port": 9200,
"scheme": "http",
"method": "get",
"path": "/_xpack/ml/anomaly_detectors/total-requests/results/buckets",
"body": "{ "anomaly_score": 75, "start": "" }"
},
"status_code": 400
}
},
"condition": {
"type": "always",
"status": "success",
"met": true
},
"transform": {
"type": "script",
"status": "success",
"payload": {
"time": "2019-03-21T15:05:40.768Z"
}
}
}

As it can be seen the script transformed the date value successfully, but the templating cannot insert it ("failed to parse date field with format [strict_date_optional_time||epoch_millis]"). If I copy the ctx.payload.time value and insert it into the body property, all works well.

So this seems to be a bug, unless I'm missing something.
What do you think?

It looks to me as if the HTTP request the input has sent failed with a HTTP error code 400. Have you verified that one is working as expected, when calling via curl or postman or any other HTTP client?

--Alex

Hi Alex!

The HTTP request to the endpoint "/_xpack/ml/anomaly_detectors/total-requests/results/buckets" is failing, because the start date cannot be empty, that's why the 400 status code is returned.

See the actual request body:
"body": "{ "anomaly_score": 75, "start": "" }"

And that's, because this variable substitution does not happen:
"body": "{ "anomaly_score": 75, "start": "{{ctx.payload.time}}" }"

I see. The transform is only executed after the condition, which in turn is executed after the input, where you are trying to access this field.

You could use a chain input, where you first action is a transform script to modify the execution time like you did in your transform and then use that time in the second part of your chain input, which would be the HTTP request.

See https://www.elastic.co/guide/en/elastic-stack-overview/6.6/input-chain.html#_transforming_chained_input_data

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.