Shard fails because of java.lang.IllegalStateException: A document doesn't have a value for a field!

Elastic Stack Kibana
1

Hi,

I am using Elasticsearch and Kibana in version 7.6.1, which I migrated from 6.8.5
I have a scripted field:

// user
String path = doc['logmessage.keyword'].value;
String logger = doc['logger.keyword'].value;
    if (path != null && path.toLowerCase().contains("user")) {
      if ("ClientSessionInitializer".equals(logger)) {
        int firstIndex = path.indexOf("'");
        int lastIndex = path.lastIndexOf("'");
        if (lastIndex > 0) {
            return path.substring(firstIndex+1,lastIndex);
          }
      } 
      if ("PermissionManager".equals(logger)) {
        int lastColon = path.lastIndexOf(":");
        if (lastColon > 0) {
            return path.substring(lastColon+1);
          }
      } 
       if ("LdapAuthenticationModule".equals(logger)) {
        int lastColon = path.lastIndexOf(":");
        if (lastColon > 0) {
            return path.substring(lastColon+1);
          }
      } 

    }
    return "";

which worked fine in verison 6.8, but now causes failing shards in Discovery:

[2020-03-20T16:18:10,603][DEBUG][o.e.a.s.TransportSearchAction] [Ada Lovelace] [531715] Failed to execute fetch phase
org.elasticsearch.transport.RemoteTransportException: [Ada Lovelace][6.204.132.58:9300][indices:data/read/search[phase/fetch/id]]
Caused by: org.elasticsearch.script.ScriptException: runtime error
        at org.elasticsearch.painless.PainlessScript.convertToScriptException(PainlessScript.java:94) ~[?:?]
        at org.elasticsearch.painless.PainlessScript$Script.execute(// user ...:837) ~[?:?]
        at org.elasticsearch.search.fetch.subphase.ScriptFieldsPhase.hitsExecute(ScriptFieldsPhase.java:67) ~[elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:178) ~[elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.search.SearchService.lambda$executeFetchPhase$5(SearchService.java:507) ~[elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:58) [elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:73) ~[elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) [elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:692) [elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.6.1.jar:7.6.1]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:830) [?:?]
Caused by: java.lang.IllegalStateException: A document doesn't have a value for a field! Use doc[<field>].size()==0 to check if a document is missing a field!
        at org.elasticsearch.index.fielddata.ScriptDocValues$Strings.get(ScriptDocValues.java:496) ~[elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.index.fielddata.ScriptDocValues$Strings.getValue(ScriptDocValues.java:503) ~[elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.painless.PainlessScript$Script.execute(// user ...:48) ~[?:?]
        at org.elasticsearch.search.fetch.subphase.ScriptFieldsPhase.hitsExecute(ScriptFieldsPhase.java:67) ~[elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:178) ~[elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.search.SearchService.lambda$executeFetchPhase$5(SearchService.java:507) ~[elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:58) ~[elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:73) ~[elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) ~[elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:692) ~[elasticsearch-7.6.1.jar:7.6.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.6.1.jar:7.6.1]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
        at java.lang.Thread.run(Thread.java:830) ~[?:?]

How would I debug this?

Best regards,
Robert

2

Hi @leprovokateur,

Based on this error it sounds like maybe you need to add another check to make sure you don't hit a situation where path or logger may be empty (I see path != null, but is it possible logger is ever empty?)

Would you be able to provide a sample document to give a better idea as to the data you are working with?

3

Hi,

The documents differ quite often, but logger is always filled with a value.

Would it be possible to have the missing field printed in the stacktrace?

Best regards,
Robert

4

Hi,

It's impossible to find a document, where the shard fails, as kibana doesn't show anything in this case and the elasticsearch log doesn't help either.

How would I find out, which field, index or document causes this error?
What am I supposed to do, if the size of a field is 0? Return 0, "", or nothing at all?

Best regards,
Robert

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.