Watcher / Deprecation: [types removal] Specifying types in document index requests is deprecated, use the typeless endpoints instead (/{index}/_doc/{id}, /{index}/_doc, or /{index}/_create/{id})

Elastic Stack Elasticsearch
1

Hello,

When trying to insert a watcher in ElasticSearch:

PUT /watcher/watch/pricealert
{
"watch":
{
"trigger": {
"schedule": {
"interval": "1m"
}
},
"input": {
"chain": {
"inputs": [
{
"first": {
"search": {
"request": {
"indices": [
"price-request-2020.01.28"
],
"body": {
"query": {
"match_all": {}
}
}
}
}
}
},
{
"second": {
"search": {
"request": {
"indices": [
"mktdata-2020.01.28"
],
"body": {
"query": {
"match_all": {}
}
}
}
}
}
}
]
}
},
"condition": {
"script": {
"source": "return ctx.payload.first.hits.hits.0._source.spot >= ctx.payload.second.hits.hits.0._source.lbound && ctx.payload.first.hits.hits.0._source.spot <= ctx.payload.second.hits.hits.0._source.ubound",
"lang": "painless"
}
},
"actions": {
"log": {
"logging": {
"level": "info",
"text": "spot out of bound!"
}
}
}
}
}

I met this problem:
#! Deprecation: [types removal] Specifying types in document index requests is deprecated, use the typeless endpoints instead (/{index}/_doc/{id}, /{index}/_doc, or /{index}/_create/{id}).
{
"_index" : "watcher",
"_type" : "watch",
"_id" : "pricealert",
"_version" : 2,
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 14,
"_primary_term" : 1
}

Does anyone know from where does it come from?

I try to compare three fields ( one field "spot" from "price-request-2020.01.28" indice with the other two: "lbound" and "ubound" from "mktdata-2020.01.28" indice)
This is what look like the "mktdata-2020.01.28" indice :
"hits" : [
{
"_index" : "mktdata-2020.01.28",
"_type" : "_doc",
"_id" : "c8Ze628BDeRluIZMFHSD",
"_score" : 1.0,
"_source" : {
"twod" : 5.0,
"@timestamp" : "2020-01-28T08:56:48.165Z",
"spot" : 1.0,
"oned" : 3.0
}

Thanks
Best regards
Pierre Jutard

2

I went through this problem by deleting the:
{
"watch":
{
at the begining of my watch configuration.

Best,
Pierre Jutard

3

This looks however as if you are missing an underscore the beginning of the URL, i.e. like _watcher

The request you send simply creates an watcher index, but is not creating a watch using the put watch API.

4

Yes you' re right Alex.

But even with the "_" the watcher is not created.

With this watcher configuration:

PUT _watcher/watch/pricealert
{
"trigger": {
"schedule": {
"interval": "1m"
}
},
"input": {
"chain": {
"inputs": [
{
"first": {
"search": {
"request": {
"indices": [
"price-request-2020.01.28"
],
"body": {
"size": 10,
"query": {
"match_all": {}
}
}
}
}
}
},
{
"second": {
"search": {
"request": {
"indices": [
"mktdata-2020.01.28"
],
"body": {
"size": 10,
"query": {
"match_all": {}
}
}
}
}
}
}
]
}
},
"condition": {
"script": {
"source": "return ctx.payload.first.hits.hits.0._source.spot >= 0.8",
"lang": "painless"
}
},
"actions": {
"log": {
"logging": {
"level": "info",
"text": "spot out of bound!"
}
}
}
}

I get this result:

{
"_id" : "pricealert",
"_version" : 13,
"_seq_no" : 12,
"_primary_term" : 1,
"created" : false
}

Do you know why?

When running with POST method this watch i have also this result:
{
"_id" : "pricealert_77888106-c314-4075-a55e-6a9066ca9d71-2020-01-28T13:13:57.142Z",
"watch_record" : {
"watch_id" : "pricealert",
"node" : "p-BVjNRJT_WIpKDIrre42Q",
"state" : "failed",
"status" : {
"state" : {
"active" : true,
"timestamp" : "2020-01-28T13:10:54.115Z"
},
"actions" : {
"log" : {
"ack" : {
"timestamp" : "2020-01-28T13:10:54.115Z",
"state" : "awaits_successful_execution"
}
}
},
"execution_state" : "failed",
"version" : 13
},
"trigger_event" : {
"type" : "manual",
"triggered_time" : "2020-01-28T13:13:57.142Z",
"manual" : {
"schedule" : {
"scheduled_time" : "2020-01-28T13:13:57.142Z"
}
}
},
"input" : {
"chain" : {
"inputs" : [
{
"first" : {
"search" : {
"request" : {
"search_type" : "query_then_fetch",
"indices" : [
"price-request-2020.01.28"
],
"rest_total_hits_as_int" : true,
"body" : {
"size" : 10,
"query" : {
"match_all" : { }
}
}
}
}
}
},
{
"second" : {
"search" : {
"request" : {
"search_type" : "query_then_fetch",
"indices" : [
"mktdata-2020.01.28"
],
"rest_total_hits_as_int" : true,
"body" : {
"size" : 10,
"query" : {
"match_all" : { }
}
}
}
}
}
}
]
}
},
"condition" : {
"script" : {
"source" : "return ctx.payload.first.hits.hits.0._source.spot >= 0.8",
"lang" : "painless"
}
},
"result" : {
"execution_time" : "2020-01-28T13:13:57.142Z",
"execution_duration" : 3,
"input" : {
"type" : "chain",
"status" : "success",
"payload" : {
"first" : {
"_shards" : {
"total" : 1,
"failed" : 0,
"successful" : 1,
"skipped" : 0
},
"hits" : {
"hits" : [
{
"_index" : "price-request-2020.01.28",
"_type" : "_doc",
"_source" : {
"@timestamp" : "2020-01-28T08:57:34.000Z",
"subject" : "price request",
"lbound" : 0.8,
"ubound" : 1.2
},
"_id" : "dMZe628BDeRluIZM-HR9",
"_score" : 1.0
}
],
"total" : 1,
"max_score" : 1.0
},
"took" : 1,
"timed_out" : false
},
"second" : {
"_shards" : {
"total" : 1,
"failed" : 0,
"successful" : 1,
"skipped" : 0
},
"hits" : {
"hits" : [
{
"_index" : "mktdata-2020.01.28",
"_type" : "_doc",
"_source" : {
"@timestamp" : "2020-01-28T08:56:48.165Z",
"spot" : 1.0,
"oned" : 3.0,
"twod" : 5.0
},
"_id" : "c8Ze628BDeRluIZMFHSD",
"_score" : 1.0
}
],
"total" : 1,
"max_score" : 1.0
},
"took" : 0,
"timed_out" : false
}
},
"chain" : {
"first" : {
"type" : "search",
"status" : "success",
"payload" : {
"_shards" : {
"total" : 1,
"failed" : 0,
"successful" : 1,
"skipped" : 0
},
"hits" : {
"hits" : [
{
"_index" : "price-request-2020.01.28",
"_type" : "_doc",
"_source" : {
"@timestamp" : "2020-01-28T08:57:34.000Z",
"subject" : "price request",
"lbound" : 0.8,
"ubound" : 1.2
},
"_id" : "dMZe628BDeRluIZM-HR9",
"_score" : 1.0
}
],
"total" : 1,
"max_score" : 1.0
},
"took" : 1,
"timed_out" : false
},
"search" : {
"request" : {
"search_type" : "query_then_fetch",
"indices" : [
"price-request-2020.01.28"
],
"rest_total_hits_as_int" : true,
"body" : {
"size" : 10,
"query" : {
"match_all" : { }
}
}
}
}
},
"second" : {
"type" : "search",
"status" : "success",
"payload" : {
"_shards" : {
"total" : 1,
"failed" : 0,
"successful" : 1,
"skipped" : 0
},
"hits" : {
"hits" : [
{
"_index" : "mktdata-2020.01.28",
"_type" : "_doc",
"_source" : {
"@timestamp" : "2020-01-28T08:56:48.165Z",
"spot" : 1.0,
"oned" : 3.0,
"twod" : 5.0
},
"_id" : "c8Ze628BDeRluIZMFHSD",
"_score" : 1.0
}
],
"total" : 1,
"max_score" : 1.0
},
"took" : 0,
"timed_out" : false
},
"search" : {
"request" : {
"search_type" : "query_then_fetch",
"indices" : [
"mktdata-2020.01.28"
],
"rest_total_hits_as_int" : true,
"body" : {
"size" : 10,
"query" : {
"match_all" : { }
}
}
}
}
}
}
},
"actions" :
},
"exception" : {
"type" : "script_exception",
"reason" : "runtime error",
"script_stack" : [
"return ctx.payload.first.hits.hits.0._source.spot >= 0.8",
" ^---- HERE"
],
"script" : "return ctx.payload.first.hits.hits.0._source.spot >= 0.8",
"lang" : "painless",
"caused_by" : {
"type" : "null_pointer_exception",
"reason" : null,
"stack_trace" : """java.lang.NullPointerException
at org.elasticsearch.painless.DefBootstrap$MIC.checkLHS(DefBootstrap.java:385)
at org.elasticsearch.painless.PainlessScript$Script.execute(return ctx.payload.first.hits.hits.0._source.spot >= 0.8:45)
at org.elasticsearch.xpack.watcher.condition.ScriptCondition.doExecute(ScriptCondition.java:60)
at org.elasticsearch.xpack.watcher.condition.ScriptCondition.execute(ScriptCondition.java:55)
at org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:513)
at org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:320)
at org.elasticsearch.xpack.watcher.transport.actions.execute.TransportExecuteWatchAction$1.doRun(TransportExecuteWatchAction.java:159)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at org.elasticsearch.xpack.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:627)
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:703)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
"""
},
"stack_trace" : """ScriptException[runtime error]; nested: NullPointerException;
at org.elasticsearch.painless.PainlessScript.convertToScriptException(PainlessScript.java:94)
at org.elasticsearch.painless.PainlessScript$Script.execute(return ctx.payload.first.hits.hits.0._source.spot >= 0.8:45)
at org.elasticsearch.xpack.watcher.condition.ScriptCondition.doExecute(ScriptCondition.java:60)
at org.elasticsearch.xpack.watcher.condition.ScriptCondition.execute(ScriptCondition.java:55)
at org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:513)
at org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:320)
at org.elasticsearch.xpack.watcher.transport.actions.execute.TransportExecuteWatchAction$1.doRun(TransportExecuteWatchAction.java:159)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at org.elasticsearch.xpack.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:627)
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:703)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NullPointerException
at org.elasticsearch.painless.DefBootstrap$MIC.checkLHS(DefBootstrap.java:385)
... 12 more
"""
}
}
}

So my question is : Is a watch not created when there is an error in my configuration?

Best,
Pierre Jutard

5

What Elasticsearch version is this?

6

7.5.2 basic license with 30 day trial extension for the watcher use.

7

Three things.

First, please format your snippets. You can use markdown in here, and this is super hard to read, especially JSON or exceptions. Thanks.

Second, the PUT Watch API response from above with the pricealert executed as expected and was stored as a watch.

Third, the third was the output of the exeucte watch API I suppose and has nothing to do with storing the watch. The exception you can read there shows that you tried to access a field, that did not exist. The script is wrong, as you used mustache syntax. Try ctx.payload.first.hits.hits[0]._source.spot - also check the result as I do not see a spot field in the first payload, but I may be missing it due to the formatting.

hope this helps

8

Sorry for the first....

For the second, i understand that this is just updated and not created.

For the third, you're right i have inverted the second with the first.
But even by following your syntax and make the inversion, it is not working.

This is the result of my two indices that are in the chain inputs:
First:

image
image1471×699 30.8 KB

Second:

image
image1464×688 29.6 KB

My configuration watch:

image
image1217×658 20.9 KB

image
image785×599 9.56 KB

image
image808×217 3.4 KB

The result:

image
image805×696 16.9 KB

image
image780×697 10.7 KB

image
image781×691 11.6 KB

image
image588×698 9.15 KB

image
image630×695 11.5 KB

image
image659×702 10.7 KB

image
image811×686 17.5 KB

image
image792×686 34.5 KB

image
image789×669 36.5 KB

Is it a size problem of my search?

Thank you,
Pierre Jutard

9

Yes, it' s a size problem, when setting Size:10 it worked and i go to a cast error after but when you change the equality in inequality it worked definitly.
Final solution:

image
image581×646 7.89 KB

image
image1920×1080 267 KB

image

Thanks
Pierre Jutard

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.