Unable to use ctx.payload.hits.hits.0._source in transform script

Elastic Stack Elasticsearch
1

Can anyone tell me why I would not be able to use ctx.payload.hits.hits.index._source in a script? I get the bellow error when I try to assign it to a var ctx.vars.test1. However, when I do ctx.payload.hits and assign it to a var everything works fine.

{"error":"WatcherException[failed to put watch [log_error_watch]]; nested: ScriptTransformValidationException[failed to compile script [test] with lang [groovy] of type [file]]; nested: ElasticsearchIllegalArgumentException[Unable to find on disk script test]; ","status":500}

curl -XPUT 'http://localhost:9200/_watcher/watch/log_error_watch' -d '{
  "trigger": {
    "schedule": {
      "interval": "10s"
    }
  },

  "input": {
    "search": {
      "request": {
        "indices": [
          "logstash-*"
        ],
        "body": {
          "query": {
            "filtered": {
              "query": {
                "bool": {
                  "must": [
                    {
                      "match": {
                        "message": "WIN7-64-COMP"
                      }
                    },
                    {
                      "match": {
                        "tags": "ad-logon-failure"
                      }
                    },
                    {
                      "range": {
                        "@timestamp": {
                          "gt": "2015-07-15T15:23:36.430"
                        }
                      }
                    }
                  ]
                }
              }
            }
          }
        }
      }
    }
  },


"transform" : { "script" : { "file" : "test" }},



  "condition": {
    "compare": {
      "ctx.payload.hits.total": {
        "gt": 0
      }
    }
  },




   "actions" : {
   "log_error" : {"logging" : {"text" : "Found {{ctx.payload}} errors in logs {{ctx.vars.test1}} and {{ctx.vars.test2}}"}}
  }

}'

Here is some additional info from elasticsearch logs:

[2015-07-15 13:40:32,661][INFO ][watcher ] [Norrin Radd] watch service has started
[2015-07-15 13:40:51,785][WARN ][watcher ] [Norrin Radd] failed to put watch [log_error_watch]
org.elasticsearch.watcher.transform.script.ScriptTransformValidationException: failed to compile script [test] with lang
[groovy] of type [file]
at org.elasticsearch.watcher.transform.script.ExecutableScriptTransform.(ExecutableScriptTransform.java:50)
at org.elasticsearch.watcher.transform.script.ScriptTransformFactory.createExecutable(ScriptTransformFactory.java
:54)
at org.elasticsearch.watcher.transform.script.ScriptTransformFactory.createExecutable(ScriptTransformFactory.java
:32)
at org.elasticsearch.watcher.transform.TransformFactory.parseExecutable(TransformFactory.java:53)
at org.elasticsearch.watcher.transform.TransformRegistry.parse(TransformRegistry.java:62)
at org.elasticsearch.watcher.transform.TransformRegistry.parse(TransformRegistry.java:51)
at org.elasticsearch.watcher.watch.Watch$Parser.parse(Watch.java:308)
at org.elasticsearch.watcher.watch.Watch$Parser.parse(Watch.java:272)
at org.elasticsearch.watcher.watch.Watch$Parser.parseWithSecrets(Watch.java:258)
at org.elasticsearch.watcher.WatcherService.putWatch(WatcherService.java:138)
at org.elasticsearch.watcher.transport.actions.put.TransportPutWatchAction.masterOperation(TransportPutWatchActio
n.java:68)
at org.elasticsearch.watcher.transport.actions.put.TransportPutWatchAction.masterOperation(TransportPutWatchActio
n.java:39)
at org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.run(TransportMasterNodeOperationA
ction.java:134)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.elasticsearch.ElasticsearchIllegalArgumentException: Unable to find on disk script test
at org.elasticsearch.script.ScriptService.compileInternal(ScriptService.java:308)
at org.elasticsearch.script.ScriptService.compile(ScriptService.java:287)
at org.elasticsearch.script.ScriptService.compile(ScriptService.java:253)
at org.elasticsearch.watcher.support.init.proxy.ScriptServiceProxy.compile(ScriptServiceProxy.java:55)
at org.elasticsearch.watcher.support.init.proxy.ScriptServiceProxy.compile(ScriptServiceProxy.java:59)
at org.elasticsearch.watcher.transform.script.ExecutableScriptTransform.(ExecutableScriptTransform.java:48)
... 15 more

1 Like
2

Hi Chris,

The error isn't about whether you can use the mentioned expression, but it can't find the script file test in the script directory ($ES_HOME/config/scripts). If you use file based scripts, your scripts need to be available in the script directories of all master and data nodes in your cluster.

This scripting page tells you more about inline, file based and index based scripts in ES:
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-scripting.html

Martijn

3

Hey Martijn,

thanks for the reply. I have my script in the correct location as I ran what I posted above before when I left out ctx.payload.hits.hits.0._source. The script isn't getting compiled so I am assuming thats why it can't find the file.

I did find figure out that you can store ctx.payload.hits.hits in a variable and loop through it that way but I'm still lost as to why ct.payload.hits.hits.index# will not work in my groovy script.