Mustache throwing NoClassDefFoundError

Elastic Stack Elasticsearch
1

Hi,

I am writing a java plugin which uses Mustache and NoClassDefFoundError exception is thrown by ElasticSearch, When I am trying to create an instance of class "MustacheScriptEngineService".

final ScriptEngineService engine = new MustacheScriptEngineService();

Elastic Search version :5.4.1

I am using maven and i have added following dependencies in pom.xml

org.elasticsearch.plugin
lang-mustache-client
5.4.1

Build succeeds but NoClassDefFoundError is thrown at runtime when i try to access java plugin.
ES stack trace:

java.lang.NoClassDefFoundError: org/elasticsearch/script/mustache/MustacheScriptEngineService
        at com.microsoft.search.script.CreatePhraseSuggesterQuery.runAsDouble(CreatePhraseSuggesterQuery.java:76) ~[?:?]
        at org.elasticsearch.script.AbstractDoubleSearchScript.run(AbstractDoubleSearchScript.java:30) ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.elasticsearch.search.aggregations.support.values.ScriptDoubleValues.setDocument(ScriptDoubleValues.java:46) ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.elasticsearch.search.MultiValueMode$5.pick(MultiValueMode.java:424) ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.elasticsearch.search.MultiValueMode$10.get(MultiValueMode.java:647) ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.elasticsearch.search.aggregations.metrics.max.MaxAggregator$1.collect(MaxAggregator.java:88) ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.elasticsearch.search.aggregations.LeafBucketCollector.collect(LeafBucketCollector.java:82) ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.apache.lucene.search.MultiCollector$MultiLeafCollector.collect(MultiCollector.java:174) ~[lucene-core-6.5.1.jar:6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - jimczi - 2017-04-21 12:17:15]
        at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:246) ~[lucene-core-6.5.1.jar:6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - jimczi - 2017-04-21 12:17:15]
        at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:197) ~[lucene-core-6.5.1.jar:6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - jimczi - 2017-04-21 12:17:15]
        at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39) ~[lucene-core-6.5.1.jar:6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - jimczi - 2017-04-21 12:17:15]
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:669) ~[lucene-core-6.5.1.jar:6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - jimczi - 2017-04-21 12:17:15]
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:473) ~[lucene-core-6.5.1.jar:6.5.1 cd1f23c63abe03ae650c75ec8ccb37762806cc75 - jimczi - 2017-04-21 12:17:15]
        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:388) ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:108) ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:247) ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:262) ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:339) ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:336) ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:627) ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.4.1.jar:5.4.1]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_161]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_161]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
Caused by: java.lang.ClassNotFoundException: org.elasticsearch.script.mustache.MustacheScriptEngineService
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_161]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_161]
        at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:814) ~[?:1.8.0_161]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_161]

Other plugins seems to work fine.
Am i missing something here? Do i need to add any other dependencies or configurations to use 'MustacheScriptEngineService'?

Thanks

2

Hey,

I am not a hundred percent sure how this has happened, without seeing more of the code/build process.

Have you considered using the ScriptService just create a mustache based Script und use the service for compilation?

--Alex

3

Thanks for the reply. I tried looking into ScriptService but couldnt find anything relevant to my issue.
Could you please give a example or point me to some documentation/code?

What i am essentially trying to achieve here is:
I want to use inline for Phrase suggester collate query. but due to security reasons we have disabled inline, hence i am trying to write a native java plugin for the same.
Is this the correct approach?
Is there any other way to achieve this without enabling script.inline?

Any insights regarding this will be really helpful.

Thanks

4

Hey,

I haven't tested it, but you could try out setting the template script context to be allowed, see https://www.elastic.co/guide/en/elasticsearch/reference/6.2/modules-scripting-security.html#allowed-script-contexts-setting

--Alex

5

I was talking about 6.x functionality here, misread that you are using 5.x. Sorry for the confusion.

Not sure if you want to enable script.engine.mustache.inline: true

6

Thanks. But, No. I dont want to enable inline scripts.
In that case, is there any alternative to use collate queries for Phrase suggester?

Also, can i use native scripts in collate queries?

7

you should be able to do sth like this

        "collate": {
          "query": {
            "lang" : "what_ever_your_script_engine_implements",
            "source" : "my_scripting_code"
          },

see https://www.elastic.co/guide/en/elasticsearch/reference/5.5/modules-scripting-engine.html

native scripts have been removed in 6.x in favor of custom script engines

8

Hi,

Thanks a lot for your suggestion. I tried above collate query and it works fine.
I would still need to take a dependency on MustacheScriptEngineService class, which is again throwing 'NoClassDefFoundError'.

I want to refer this service in my plugin and i am using maven for the build process.
Following is the details of the dependecies:
lang-mustache-5.4.1 : located at ES\elasticsearch-5.4.1\modules\lang-mustache
lang-mustache-client-5.4.1 : located at maven classpath (.m2\repository\org\elasticsearch\plugin\lang-mustache-client\5.4.1)

pom.xml includes the mustache dependency as mentioned earlier. Build is successful but i am running into 'NoClassDefFoundError' exception.
Please let me know if i am missing any dependencies or config.
Any suggestions to fix this issue would be really helpful.

Thanks

9

The client lib in this case is just a copy of lang-mustache with a different name (with the eventual intention to limit the client to only those classes needed by the transport client).

How did you install elasticsearch and how are you starting it?

10

So then both jar would contain the class 'MustacheScriptEngineService'.?
Then i am not clear why is it throwing theexception at runtime?
I downloaded windows 64bit ES 5.4.1 zip file , unzipped it and just running the elasticsearch.bat file.

Thanks,
Ishita

11

What are all the jar files you have on the classpath for your client?

12

Do you mean lib folder of Elasticsearch?

Following is the list of jars present in lib folder

elasticsearch-5.4.1.jar           
HdrHistogram-2.1.9.jar            
hppc-0.7.1.jar                    
jackson-core-2.8.6.jar            
jackson-dataformat-cbor-2.8.6.jar 
jackson-dataformat-smile-2.8.6.jar
jackson-dataformat-yaml-2.8.6.jar                   
java-version-checker-5.4.1.jar    
jna-4.4.0.jar                     
joda-time-2.9.5.jar               
jopt-simple-5.0.2.jar             
jts-1.13.jar                      
log4j-1.2-api-2.8.2.jar           
log4j-api-2.8.2.jar               
log4j-core-2.8.2.jar              
lucene-analyzers-common-6.5.1.jar 
lucene-backward-codecs-6.5.1.jar  
lucene-core-6.5.1.jar             
lucene-grouping-6.5.1.jar         
lucene-highlighter-6.5.1.jar      
lucene-join-6.5.1.jar             
lucene-memory-6.5.1.jar           
lucene-misc-6.5.1.jar             
lucene-queries-6.5.1.jar          
lucene-queryparser-6.5.1.jar      
lucene-sandbox-6.5.1.jar          
lucene-spatial-6.5.1.jar          
lucene-spatial-extras-6.5.1.jar   
lucene-spatial3d-6.5.1.jar        
lucene-suggest-6.5.1.jar          
securesm-1.1.jar                  
snakeyaml-1.15.jar                
spatial4j-0.6.jar                 
t-digest-3.0.jar

I tried adding lang-mustache-5.4.1/lang-mustache-client-5.4.1 in lib folder which gave "jar hell" error.
Also i couldn't remove this jar from Modules folder, because otherwise Elasticsearch wont start successfully.

13

I mean the jars for your application. Can you confirm you:

  1. Start elasticsearch using bin/elasticsearch
  2. Start your user application with the elasticsearch transport client jar (+ transitive dependencies) and mustache client jar (+ transitive dependencies)
  3. Connect to the elasticsearch service by creating a transport client, configured to the service you started in step 1
14

Hi,

I think i am listening to transport client only via IntelliJ.

D:\OldES\elasticsearch-5.4.1\bin>.\elasticsearch.bat
Listening for transport dt_socket at address: 5005
[2018-04-24T11:19:12,502][INFO ][o.e.n.Node ] [] initializing ...
[2018-04-24T11:19:12,618][INFO ][o.e.e.NodeEnvironment ] [YYBkre7] using [1] data paths, mounts [[DATADRIVE1 (D:)]], net usable_space [1.6tb], net total_space [1.8tb], spins? [unknown], types [NTFS]
[2018-04-24T11:19:12,618][INFO ][o.e.e.NodeEnvironment ] [YYBkre7] heap size [1.9gb], compressed ordinary object pointers [true]
[2018-04-24T11:19:13,972][INFO ][o.e.n.Node ] node name [YYBkre7] derived from node ID [YYBkre7ERAuSjfLmM1yk7w]; set [node.name] to override
[2018-04-24T11:19:13,973][INFO ][o.e.n.Node ] version[5.4.1], pid[23836], build[2cfe0df/2017-05-29T16:05:51.443Z], OS[Windows 10/10.0/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_161/25.161-b12]
[2018-04-24T11:19:14,034][INFO ][o.e.n.Node ] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+DisableExplicitGC, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005, -Delasticsearch, -Des.path.home=D:\OldES\elasticsearch-5.4.1]
[2018-04-24T11:19:15,761][INFO ][o.e.p.PluginsService ] [YYBkre7] loaded module [aggs-matrix-stats]
[2018-04-24T11:19:15,762][INFO ][o.e.p.PluginsService ] [YYBkre7] loaded module [ingest-common]
[2018-04-24T11:19:15,762][INFO ][o.e.p.PluginsService ] [YYBkre7] loaded module [lang-expression]
[2018-04-24T11:19:15,762][INFO ][o.e.p.PluginsService ] [YYBkre7] loaded module [lang-groovy]
[2018-04-24T11:19:15,762][INFO ][o.e.p.PluginsService ] [YYBkre7] loaded module [lang-mustache]
[2018-04-24T11:19:15,763][INFO ][o.e.p.PluginsService ] [YYBkre7] loaded module [lang-painless]
[2018-04-24T11:19:15,763][INFO ][o.e.p.PluginsService ] [YYBkre7] loaded module [percolator]
[2018-04-24T11:19:15,764][INFO ][o.e.p.PluginsService ] [YYBkre7] loaded module [reindex]
[2018-04-24T11:19:15,764][INFO ][o.e.p.PluginsService ] [YYBkre7] loaded module [transport-netty3]
[2018-04-24T11:19:15,764][INFO ][o.e.p.PluginsService ] [YYBkre7] loaded module [transport-netty4]
[2018-04-24T11:19:15,765][INFO ][o.e.p.PluginsService ] [YYBkre7] loaded plugin [AlmSearchPlugin]
[2018-04-24T11:19:17,785][INFO ][o.e.d.DiscoveryModule ] [YYBkre7] using discovery type [zen]
[2018-04-24T11:19:18,227][WARN ][o.e.c.u.IndexFolderUpgrader] [D:\OldES\elasticsearch-5.4.1\data\nodes\0\indices\FBcUdCTwSaCrm46WlkHhww] no index state found - ignoring
[2018-04-24T11:19:18,228][WARN ][o.e.c.u.IndexFolderUpgrader] [D:\OldES\elasticsearch-5.4.1\data\nodes\0\indices\Upt94CD6Saipvah_xcDYKQ] no index state found - ignoring
[2018-04-24T11:19:18,247][WARN ][o.e.c.u.IndexFolderUpgrader] [D:\OldES\elasticsearch-5.4.1\data\nodes\0\indices\skUK7KPLQ4uHoGFwKQfc7Q] no index state found - ignoring
[2018-04-24T11:19:18,269][WARN ][o.e.c.u.IndexFolderUpgrader] [D:\OldES\elasticsearch-5.4.1\data\nodes\0\indices\NwPvPs9MSwmcS_v9J3r8Ng] no index state found - ignoring
[2018-04-24T11:19:18,270][WARN ][o.e.c.u.IndexFolderUpgrader] [D:\OldES\elasticsearch-5.4.1\data\nodes\0\indices\bhdl-mqLSly-kdwvTo0vPQ] no index state found - ignoring
[2018-04-24T11:19:18,448][INFO ][o.e.n.Node ] initialized
[2018-04-24T11:19:18,449][INFO ][o.e.n.Node ] [YYBkre7] starting ...
[2018-04-24T11:19:18,869][INFO ][o.e.t.TransportService ] [YYBkre7] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}, {[::1]:9300}
[2018-04-24T11:19:21,923][INFO ][o.e.c.s.ClusterService ] [YYBkre7] new_master {YYBkre7}{YYBkre7ERAuSjfLmM1yk7w}{o-ocAz7_SyyG5i_kuzsIwg}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2018-04-24T11:19:22,104][INFO ][o.e.h.n.Netty4HttpServerTransport] [YYBkre7] publish_address {127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}, {[::1]:9200}
[2018-04-24T11:19:22,107][INFO ][o.e.n.Node ] [YYBkre7] started

And later i am attaching the debugger to the service started in step 1

I am not very sure what do you mean by step 2.
Could you please clarify?

Thanks,
Ishita

15

Hi,

I was able to figure out a workaround for this issue.

If i delete the "lang-mustache" folder inside "modules" and place the required jar file (lang-mustache-client-5.4.1) in lib folder, then i was able to successfully create a instance of MustacheScriptEngineService class and use it.

This is definitely not a valid solution.
Also, i am not able to understand why was i able to refer a dependency present in "lib" folder and not from "lang-mustache"?

Any help solving this issue would be really appreciated!

Thanks

16

Sorry for the delayed reply.

Why are you trying to construct MustacheScriptEngineService? This class is used internally by plugin initialization, but that is not how to use mustache from a client.

Looking back at the original issue here, it looks like you are trying to write a plugin (sorry I missed this before). You should not be using a transport client. Instead, your plugin needs to use the ScriptService to compile a script, and then execute it using the script apis.

17

Hi,
I couldnt find a way where i can leverage mustache template language by using ScriptService.
Could you please point me to an example or any document/discussion ?

18

Are you trying to use mustache within some new api your plugin adds? If so, you need to create ScriptContext, or use an existing one. I need more information before I can give an example how to use the script service.

19

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.