Painless syntax error

alerting

(Pjanzen) #1

Hi All,

On the below watcher I am getting a error:

Watcher: An internal server error occurred

My guess it is in the condition where I execute a piece of painless code but I cannot seem to figure out what I am doing wrong. I basicly have 2 questions.
1. If such error occures, where is this logged?
2. Could someone help me with the below watcher?

{
  "trigger": {
    "schedule": {
      "interval": "5m"
    }
  },
  "input": {
    "search": {
      "request": {
        "search_type": "query_then_fetch",
        "indices": [
          "clog-*"
        ],
        "types": [],
        "body": {
          "size": 0,
          "query": {
            "bool": {
              "must": [
                {
                  "query_string": {
                    "query": "tags:cmgw AND action:P6* AND _exists_:hdr_subject AND cm_score: [0 TO 90] AND NOT dkim:pass* AND NOT dmarc:pass*",
                    "analyze_wildcard": true
                  }
                },
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-5m",
                      "lte": "now"
                    }
                  }
                }
              ],
              "must_not": [
                {
                  "bool": {
                    "should": [
                      {
                        "match_phrase": {
                          "hdr_from": "gruppiotthon"
                        }
                      },
                      {
                        "match_phrase": {
                          "hdr_from": "gruppi.hu"
                        }
                      },
                      {
                        "match_phrase": {
                          "hdr_from": "gruppiajandek.hu"
                        }
                      },
                      {
                        "match_phrase": {
                          "hdr_from": "telekom.hu"
                        }
                      },
                      {
                        "match_phrase": {
                          "hdr_from": "szallas.hu"
                        }
                      },
                      {
                        "match_phrase": {
                          "hdr_from": "hvg.hu"
                        }
                      },
                      {
                        "match_phrase": {
                          "hdr_from": "kreativhobby.hu"
                        }
                      },
                      {
                        "match_phrase": {
                          "hdr_from": "spartoo.hu"
                        }
                      },
                      {
                        "match_phrase": {
                          "hdr_from": "vizionet.cz"
                        }
                      },
                      {
                        "match_phrase": {
                          "hdr_from": "mango.com"
                        }
                      }
                    ],
                    "minimum_should_match": 1
                  }
                }
              ]
            }
          },
          "aggs": {
            "subjectsAggs": {
              "terms": {
                "field": "hdr_subject.keyword",
                "size": 10,
                "min_doc_count": 10,
                "order": {
                  "_count": "desc"
                }
              },
              "aggs": {
                "sourceIpsAggs": {
                  "terms": {
                    "field": "ip",
                    "size": 10,
                    "order": {
                      "_count": "desc"
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "script": {
        "lang": "painless",
        "source": "if (ctx.payload.hits.total < 1) return false; for (int i=0; i < ctx.payload.aggregations.subjectsAggs.buckets.size(); i++) { if ((ctx.payload.aggregations.subjectsAggs.buckets[i].doc_count > 10) && (ctx.payload.aggregations.subjectsAggs.buckets.[i].sourceIpsAggs.[i].buckets.size() > 10)) { return true; }}",
        "params": {
            "ip.threshold": 10,
            "subject.threshold": 10
        }
    }
  },
  "actions": {
    "email_1": {
      "email": {
        "profile": "standard",
        "to": [
          "paujanzen@xxxx.com"
        ],
        "subject": "Watch [{{ctx.metadata.name}}] has exceeded the threshold",
        "body": {
          "html": "Found events : <br><br>{{ctx.payload.aggregations}}"
        }
      }
    }
  },
    "metadata": {
      "name": "eSRD"
    }
}

(Alexander Reelsen) #2

Hey,

please include the output of the execute watch API here


(Pjanzen) #3

Hi Alexander,

I would if I could actually save it as I posted it. I did manage to found the error in the log when I hit the save button.

The error:

Caused by: java.lang.IllegalArgumentException: unexpected character [[].
at org.elasticsearch.painless.PainlessScript$Script.compile(if (ctx.payload.hits.total < 1) return false; for (int i=0; i < ctx.payload.aggregations.subjectsAggs.buckets.size(); i++) { if ((ctx.payload.aggregations.subjectsAggs.buckets[i].doc_count > 10) && (ctx.payload.aggregations.subjectsAggs.buckets.[i].sourceI ...:246) ~[?:?]

The complete stacktrace.

[2018-06-27T09:06:36,754][WARN ][r.suppressed ] path: /_xpack/watcher/watch/eSRD, params: {id=eSRD}
org.elasticsearch.script.ScriptException: compile error
at org.elasticsearch.painless.PainlessScriptEngine.convertToScriptException(PainlessScriptEngine.java:538) ~[?:?]
at org.elasticsearch.painless.PainlessScriptEngine.compile(PainlessScriptEngine.java:441) ~[?:?]
at org.elasticsearch.painless.PainlessScriptEngine.compile(PainlessScriptEngine.java:148) ~[?:?]
at org.elasticsearch.script.ScriptService.compile(ScriptService.java:335) ~[elasticsearch-6.2.4.jar:6.2.4]
at org.elasticsearch.xpack.watcher.condition.ScriptCondition.(ScriptCondition.java:57) ~[?:?]
at org.elasticsearch.xpack.watcher.condition.ScriptCondition.parse(ScriptCondition.java:67) ~[?:?]
at org.elasticsearch.xpack.watcher.Watcher.lambda$createComponents$2(Watcher.java:323) ~[?:?]
at org.elasticsearch.xpack.core.watcher.condition.ConditionRegistry.parseExecutable(ConditionRegistry.java:68) ~[x-pack-core-6.2.4.jar:6.2.4]
at org.elasticsearch.xpack.watcher.watch.WatchParser.parse(WatchParser.java:157) ~[x-pack-watcher-6.2.4.jar:6.2.4]
at org.elasticsearch.xpack.watcher.watch.WatchParser.parse(WatchParser.java:123) ~[x-pack-watcher-6.2.4.jar:6.2.4]
at org.elasticsearch.xpack.watcher.watch.WatchParser.parseWithSecrets(WatchParser.java:109) ~[x-pack-watcher-6.2.4.jar:6.2.4]
at org.elasticsearch.xpack.watcher.transport.actions.put.TransportPutWatchAction.masterOperation(TransportPutWatchAction.java:84) [x-pack-watcher-6.2.4.jar:6.2.4]
at org.elasticsearch.xpack.watcher.transport.actions.put.TransportPutWatchAction.masterOperation(TransportPutWatchAction.java:59) [x-pack-watcher-6.2.4.jar:6.2.4]
at org.elasticsearch.action.support.master.TransportMasterNodeAction.masterOperation(TransportMasterNodeAction.java:88) [elasticsearch-6.2.4.jar:6.2.4]
at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$2.doRun(TransportMasterNodeAction.java:167) [elasticsearch-6.2.4.jar:6.2.4]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:672) [elasticsearch-6.2.4.jar:6.2.4]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.2.4.jar:6.2.4]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_162]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_162]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]
Caused by: java.lang.IllegalArgumentException: unexpected character [[].
at org.elasticsearch.painless.PainlessScript$Script.compile(if (ctx.payload.hits.total < 1) return false; for (int i=0; i < ctx.payload.aggregations.subjectsAggs.buckets.size(); i++) { if ((ctx.payload.aggregations.subjectsAggs.buckets[i].doc_count > 10) && (ctx.payload.aggregations.subjectsAggs.buckets.[i].sourceI ...:246) ~[?:?]
at org.elasticsearch.painless.antlr.EnhancedPainlessLexer.recover(EnhancedPainlessLexer.java:94) ~[?:?]
at org.antlr.v4.runtime.Lexer.nextToken(Lexer.java:169) ~[?:?]
at org.elasticsearch.painless.antlr.EnhancedPainlessLexer.nextToken(EnhancedPainlessLexer.java:67) ~[?:?]
at org.antlr.v4.runtime.BufferedTokenStream.fetch(BufferedTokenStream.java:185) ~[?:?]
at org.antlr.v4.runtime.BufferedTokenStream.sync(BufferedTokenStream.java:168) ~[?:?]
at org.antlr.v4.runtime.BufferedTokenStream.consume(BufferedTokenStream.java:152) ~[?:?]
at org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:556) ~[?:?]
at org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:412) ~[?:?]
at org.elasticsearch.painless.antlr.PainlessParser.unary(PainlessParser.java:1987) ~[?:?]
at org.elasticsearch.painless.antlr.PainlessParser.expression(PainlessParser.java:1647) ~[?:?]
at org.elasticsearch.painless.antlr.PainlessParser.expression(PainlessParser.java:1808) ~[?:?]
at org.elasticsearch.painless.antlr.PainlessParser.statement(PainlessParser.java:593) ~[?:?]
at org.elasticsearch.painless.antlr.PainlessParser.block(PainlessParser.java:1053) ~[?:?]
at org.elasticsearch.painless.antlr.PainlessParser.trailer(PainlessParser.java:964) ~[?:?]
at org.elasticsearch.painless.antlr.PainlessParser.statement(PainlessParser.java:773) ~[?:?]
at org.elasticsearch.painless.antlr.PainlessParser.source(PainlessParser.java:178) ~[?:?]
at org.elasticsearch.painless.antlr.Walker.buildAntlrTree(Walker.java:223) ~[?:?]
at org.elasticsearch.painless.antlr.Walker.(Walker.java:205) ~[?:?]
at org.elasticsearch.painless.antlr.Walker.buildPainlessTree(Walker.java:180) ~[?:?]
at org.elasticsearch.painless.Compiler.compile(Compiler.java:180) ~[?:?]
at org.elasticsearch.painless.PainlessScriptEngine$4.run(PainlessScriptEngine.java:429) ~[?:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_162]
at org.elasticsearch.painless.PainlessScriptEngine.compile(PainlessScriptEngine.java:425) ~[?:?]
... 18 more


(Alexander Reelsen) #4

Two things, first you are missing a return false at the end, if no aggs are matching.

Second, take a look at this

(ctx.payload.aggregations.subjectsAggs.buckets[i].doc_count > 10) && (ctx.payload.aggregations.subjectsAggs.buckets.[i].sourceIpsAggs.[i].buckets.size() > 10)) 

Looking carefully to subjectsAggs.buckets one can spot an additional dot . in there. Try removing it.


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.