Groovy Aggregations: ClassNotFoundException[sun.reflect.MethodAcces sorImpl]

Elastic Stack Elasticsearch
1

Hi there,

I was successfully running ElasticSearch 2.1.1 on my development machine running Windows, and my code is running well.

I have deployed ES 2.2 on Ubuntu 14.04.4 and migrated the existing indexes from my development machine, and started getting an error when trying to execute a query that contains scripting aggregation.

Query example:

{
  "query": {
    "filtered": {
      "query": {
        "match_all": {}
      }
    }
  },
  "aggs": {
    "myrating": {
      "scripted_metric": {
        "init_script": "_agg[\"myrating\"] = [:];",
        "map_script": "_agg.myrating.put(_source.url, _source.rank)",
        "combine_script": "max = [:]; max = _agg.myrating.sort{-it.value} ; return max;",
        "reduce_script": "max = [:]; for (shardAggs in _aggs) { max = max.plus(shardAggs) }; return max.sort{-it.value}.take(25);"
      }
    }
  }
}

Query result:

{
  "error" : {
    "root_cause" : [ {
      "type" : "script_exception",
      "reason" : "failed to run inline script [max = [:]; max = _agg.myrating.sort{-it.value} ; return max;] using lang [groovy]"
    } ],
    "type" : "search_phase_execution_exception",
    "reason" : "all shards failed",
    "phase" : "query",
    "grouped" : true,
    "failed_shards" : [ {
      "shard" : 0,
      "index" : "post-stats-2016-02",
      "node" : "XD_D4XyCRzyBT_uSd6PO7g",
      "reason" : {
        "type" : "script_exception",
        "reason" : "failed to run inline script [max = [:]; max = _agg.myrating.sort{-it.value} ; return max;] using lang [groovy]",
        "caused_by" : {
          "type" : "no_class_def_found_error",
          "reason" : "sun/reflect/MethodAccessorImpl",
          "caused_by" : {
            "type" : "class_not_found_exception",
            "reason" : "sun.reflect.MethodAccessorImpl"
          }
        }
      }
    } ]
  },
  "status" : 500
}

Related exception from ES log:

 [2016-02-29 09:48:04,330][DEBUG][action.search.type       ] [clst-es01] [post-stats-2016-02][1], node[XD_D4XyCRzyBT_uSd6PO7g], [P], v[10], s[STARTED], a[id=qvnQpRcaRM2BujYBgE8VpQ]: Failed to execute [org.elasticsearch.action.search.SearchRequest@78f6a4f] lastShard [true]
 RemoteTransportException[[clst-es01][127.0.0.1:9300][indices:data/read/search[phase/query]]]; nested: ScriptException[failed to run inline script [max = [:]; max = _agg.myrating.sort{-it.value} ; return max;] using lang [groovy]]; nested: NoClassDefFoundError[sun/reflect/MethodAccessorImpl]; nested: ClassNotFoundException[sun.reflect.MethodAccessorImpl];
 Caused by: ScriptException[failed to run inline script [max = [:]; max = _agg.myrating.sort{-it.value} ; return max;] using lang [groovy]]; nested: NoClassDefFoundError[sun/reflect/MethodAccessorImpl]; nested: ClassNotFoundException[sun.reflect.MethodAccessorImpl];
         at org.elasticsearch.script.groovy.GroovyScriptEngineService$GroovyScript.run(GroovyScriptEngineService.java:318)
         at org.elasticsearch.search.aggregations.metrics.scripted.ScriptedMetricAggregator.buildAggregation(ScriptedMetricAggregator.java:95)
         at org.elasticsearch.search.aggregations.AggregationPhase.execute(AggregationPhase.java:167)
         ... (omitted for brevity)
         at java.lang.Thread.run(Thread.java:745)
 Caused by: java.lang.NoClassDefFoundError: sun/reflect/MethodAccessorImpl
         at sun.misc.Unsafe.defineClass(Native Method)
         at sun.reflect.ClassDefiner.defineClass(ClassDefiner.java:63)
         at sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:399)
         ... (omitted for brevity)
         at java.util.TimSort.sort(TimSort.java:173)
         at java.util.Arrays.sort(Arrays.java:659)
         at java.util.Collections.sort(Collections.java:217)
         at org.codehaus.groovy.runtime.DefaultGroovyMethods.sort(DefaultGroovyMethods.java:8474)
         ... (omitted for brevity)

I have found a similar exception in a Watcher-related thread, but didn't get a hint on how to fix my error.

I suspect that this problem might have something to do with Java Security Manager, but I don't understand how to rewrite my code to fix the security problem (e.g. do I need to avoid sorting?), or what I need to add to .java.policy. The following didn't work:

superuser@esclusternode01:~$ cat /home/elasticsearch/.java.policy
grant {
    permission org.elasticsearch.script.ClassPermission "java.lang.Class";
    permission org.elasticsearch.script.ClassPermission "java.util.TimSort";
};

Any help would be appreciated!

Thank you,
Shavkat

2

As you can see from the issue, you must add

permission org.elasticsearch.script.ClassPermission "sun.reflect.MethodAccessorImpl"

to modules/lang-groovy/src/main/plugin-metadata/plugin-security.policy

1 Like
3

If that does not help, update to Groovy 2.4.6

4

Yeehaw, it's working!

Thank you very much for the reply, Jörg!

Adding

permission org.elasticsearch.script.ClassPermission "sun.reflect.MethodAccessorImpl";

to modules/lang-groovy/src/main/plugin-metadata/plugin-security.policy worked like a charm!

I wonder if this introduces any security risks going further...

Thank you!

5

Main security risk is running ES nodes with port open to the public internet, or executing unknown scripts with unknown content of unknown origin (i.e. not validating input).

The sun.reflect permissions are required for introspection, because of how meta-programming works, by inspecting JDK internals. This is not only Groovy, but the JDK itself too, for example,`sun.reflect´ is used by java.corba, java.logging, java.sql, jdk.scripting.nashorn, and also java.xml. The difference is that JDK passes all checks of security manager automatically, while external programs can not. This situation is not appropriate, andl will be addressed in Java 9.

1 Like
6

Thanks a lot for the explanation, will bear this in mind.