Scripted field throws exception

[Santhosh_ic (Santhosh Irani)]

Hi,

I have a scripted field like below.

if(doc.containsKey('Message.keyword')) {
def m = /^\s{1,}New invalid user:\s{1,}([a-zA-Z0-9]+)\s{0,}$/.matcher(doc['Message.keyword'].value);
if(m.matches()) {
return m.group(1)
} else {
return "no match"
}
}

For the existing documents, I can see the scripted field with the correct value. However, when a new event arrives below exception is thrown.

// 128s
Error: Request to Elasticsearch failed: {"error":{"root_cause":[{"type":"script_exception","reason":"runtime error","script_stack":["java.util.regex.Matcher.getTextLength(Matcher.java:1283)","java.util.regex.Matcher.reset(Matcher.java:309)","java.util.regex.Matcher.(Matcher.java:229)","java.util.regex.Pattern.matcher(Pattern.java:1093)","m = /^\s{1,}New invalid user:\s{1,}([a-zA-Z0-9]+)\s{0,}$/.matcher(doc['Message.keyword'].value);\n"," ^---- HERE"],"script":"if(doc.containsKey('Message.keyword')) {\ndef m = /^\s{1,}New invalid user:\s{1,}([a-zA-Z0-9]+)\s{0,}$/.matcher(doc['Message.keyword'].value);\nif(m.matches()) {\nreturn m.group(1)\n} else {\nreturn "no match"\n}\n}","lang":"painless"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"ws1-aw-0.6-2018.05.30","node":"BuNZuVJHSgSzFn4xbmPd0w","reason":{"type":"script_exception","reason":"runtime error","script_stack":["java.util.regex.Matcher.getTextLength(Matcher.java:1283)","java.util.regex.Matcher.reset(Matcher.java:309)","java.util.regex.Matcher.(Matcher.java:229)","java.util.regex.Pattern.matcher(Pattern.java:1093)","m = /^\s{1,}New invalid user:\s{1,}([a-zA-Z0-9]+)\s{0,}$/.matcher(doc['Message.keyword'].value);\n"," ^---- HERE"],"script":"if(doc.containsKey('Message.keyword')) {\ndef m = /^\s{1,}New invalid user:\s{1,}([a-zA-Z0-9]+)\s{0,}$/.matcher(doc['Message.keyword'].value);\nif(m.matches()) {\nreturn m.group(1)\n} else {\nreturn "no match"\n}\n}","lang":"painless","caused_by":{"type":"null_pointer_exception","reason":null}}}]},"status":500}
at http://localhost:5601/bundles/commons.bundle.js?v=16602:21:595466
at Function.Promise.try (http://localhost:5601/bundles/commons.bundle.js?v=16602:21:493783)
at http://localhost:5601/bundles/commons.bundle.js?v=16602:21:493171
at Array.map ()
at Function.Promise.map (http://localhost:5601/bundles/commons.bundle.js?v=16602:21:493129)
at callResponseHandlers (http://localhost:5601/bundles/commons.bundle.js?v=16602:21:595044)
at http://localhost:5601/bundles/commons.bundle.js?v=16602:21:584390
at processQueue (http://localhost:5601/bundles/vendors.bundle.js?v=16602:116:132456)
at http://localhost:5601/bundles/vendors.bundle.js?v=16602:116:133349
at Scope.$digest (http://localhost:5601/bundles/vendors.bundle.js?v=16602:116:144239)
at Scope.$apply (http://localhost:5601/bundles/vendors.bundle.js?v=16602:116:147018)
at done (http://localhost:5601/bundles/vendors.bundle.js?v=16602:116:100026)
at completeRequest (http://localhost:5601/bundles/vendors.bundle.js?v=16602:116:104697)
at XMLHttpRequest.xhr.onload (http://localhost:5601/bundles/vendors.bundle.js?v=16602:116:105435)

After that, for that index "Discover" always returns no result, Kibana shows "Courier Fetch: 5 of 10 shards failed" and below exception can be seen from the Elasticsearch logs

Caused by: org.elasticsearch.script.ScriptException: runtime error
at org.elasticsearch.painless.PainlessScript.convertToScriptException(PainlessScript.java:101) ~[?:?]
at org.elasticsearch.painless.PainlessScript$Script.execute(if(doc.containsKey('Message.keyword')) {
def m = /^\s{1,}New invalid user:\s{1,}([a-zA-Z0-9]+)\s{0,}$/.matcher(doc['Message.keyword'].value);
if(m.matches()) {
return m.group(1)
} else {
return "no match"
}
}:195) ~[?:?]
at org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:105) ~[?:?]
at org.elasticsearch.search.fetch.subphase.ScriptFieldsFetchSubPhase.hitsExecute(ScriptFieldsFetchSubPhase.java:
67) ~[elasticsearch-6.2.3.jar:6.2.3]
at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:170) ~[elasticsearch-6.2.3.jar:6.2.3]
at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:499) ~[elasticsearch-6.2.3.jar:6.
2.3]
at org.elasticsearch.action.search.SearchTransportService$11.messageReceived(SearchTransportService.java:440) ~[
elasticsearch-6.2.3.jar:6.2.3]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.2.3.
jar:6.2.3]
at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) [elasticsearch-6.2.3.jar:
6.2.3]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.2.3.
jar:6.2.3]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_162]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_162]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]
Caused by: java.lang.NullPointerException
at java.util.regex.Matcher.getTextLength(Matcher.java:1283) ~[?:1.8.0_162]
at java.util.regex.Matcher.reset(Matcher.java:309) ~[?:1.8.0_162]
at java.util.regex.Matcher.(Matcher.java:229) ~[?:1.8.0_162]
at java.util.regex.Pattern.matcher(Pattern.java:1093) ~[?:1.8.0_162]
at org.elasticsearch.painless.PainlessScript$Script.execute(if(doc.containsKey('Message.keyword')) {
def m = /^\s{1,}New invalid user:\s{1,}([a-zA-Z0-9]+)\s{0,}$/.matcher(doc['Message.keyword'].value);

Searching for any other text from the search bar throws below exception.

Shard Failures
The following shard failures ocurred:

Index: ws1-aw-0.6-2018.05.31 Shard: 1 Reason: {"type":"script_exception","reason":"runtime error","script_stack":["java.util.regex.Matcher.getTextLength(Matcher.java:1283)","java.util.regex.Matcher.reset(Matcher.java:309)","java.util.regex.Matcher.(Matcher.java:229)","java.util.regex.Pattern.matcher(Pattern.java:1093)","m = /^\s{1,}New invalid user:\s{1,}([a-zA-Z0-9]+)\s{0,}$/.matcher(doc['Message.keyword'].value);\n"," ^---- HERE"],"script":"if(doc.containsKey('Message.keyword')) {\ndef m = /^\s{1,}New invalid user:\s{1,}([a-zA-Z0-9]+)\s{0,}$/.matcher(doc['Message.keyword'].value);\nif(m.matches()) {\nreturn m.group(1)\n} else {\nreturn "no match"\n}\n}","lang":"painless","caused_by":{"type":"null_pointer_exception","reason":null}}

[tsullivan (Tim Sullivan)]

Hi, please keep in mind that code and error logs are way more readable if they're marked up using preformatted text. The formatting toolbar has the brackets icon for helping out with that.

The stuff that's shown after Error: Request to Elasticsearch failed: should be valid JSON, but it looks like when it got copied out of the browser, some escapes for double quotes didn't make it through. If you look at the error as it's logged from Elasticsearch, you'd be able to parse the JSON and get a helpful message. That will show you the runtime error stack and print out your code showing where there was a problem executing your script.

[Santhosh_ic (Santhosh Irani)]

I was able to solve the issue by replacing doc.containsKey('Message.keyword') with !doc['Message.keyword'].empty.