Elastic Search Deployment

Hi,
I am trying to install Elastic Stack in my local machine. I downloaded elasticsearch-8.11.3-windows-x86_64. When I try to run the batch file I get the following error message. " fatal exception while booting Elasticsearchjava.lang.IllegalStateException: failed to load plugin repository-url due to jar hell". When I searched there is a reference to older version jar file conflict. But I do not see that file in this version. "As per elasticsearch if you have elasticsearch-rest-high-level-client-6.5.1.jar file under <elastic_Root>/lib folder, delete the file and restart". Can you please help me get past this issue.

Hi @Karthikeyan_K,

Welcome!

Do you have an old install of the Elasticsearch client kicking around? It could be conflicting. Can you try removing everything and reinstalling?

Hi Carly,
I do not have any old version. This is my first installation. I also did a file search and made sure there isn't any older version on my computer.

Any other way I can find out what is conflicting?

TIA
-Karthik

HI @Karthikeyan_K,

I assume you are not able able to find /lib/elasticsearch-rest-high-level-client-6.5.1.jar anywhere? Is there any other error messages in the logs at all?

The following is what I see in the log file.

"[DESKTOP-8JCMOL0] fatal exception while booting Elasticsearch
java.lang.IllegalStateException: failed to load plugin repository-url due to jar hell
at org.elasticsearch.plugins.PluginsUtils.checkBundleJarHell(PluginsUtils.java:290) ~[elasticsearch-8.11.3.jar:?]
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:304) ~[elasticsearch-8.11.3.jar:?]
at org.elasticsearch.plugins.PluginsService.(PluginsService.java:175) ~[elasticsearch-8.11.3.jar:?]
at org.elasticsearch.plugins.PluginsService.lambda$getPluginsServiceCtor$16(PluginsService.java:725) ~[elasticsearch-8.11.3.jar:?]
at org.elasticsearch.node.Node.(Node.java:438) ~[elasticsearch-8.11.3.jar:?]
at org.elasticsearch.node.Node.(Node.java:344) ~[elasticsearch-8.11.3.jar:?]
at org.elasticsearch.bootstrap.Elasticsearch$2.(Elasticsearch.java:236) ~[elasticsearch-8.11.3.jar:?]
at org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:236) ~[elasticsearch-8.11.3.jar:?]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:73) ~[elasticsearch-8.11.3.jar:?]
Caused by: java.lang.IllegalStateException: jar hell!
class: net.jpountz.lz4.LZ4BlockInputStream
jar1: C:\elastic_stack\elasticsearch-8.11.3-windows-x86_64\elasticsearch-8.11.3\lib\lz4-java-1.8.0.jar
jar2: C:\PROGRA~1\IBM\SQLLIB\java\db2jcc4.jar
at org.elasticsearch.jdk.JarHell.checkClass(JarHell.java:315) ~[elasticsearch-core-8.11.3.jar:?]
at org.elasticsearch.jdk.JarHell.checkJarHell(JarHell.java:233) ~[elasticsearch-core-8.11.3.jar:?]
at org.elasticsearch.plugins.PluginsUtils.checkBundleJarHell(PluginsUtils.java:288) ~[elasticsearch-8.11.3.jar:?]
... 8 more"

Hi @Karthikeyan_K Welcome to the community

Are you trying to run elasticsearch as a service or just from the command line?

What documents exactly are you following?

Can you show the exact command you are using to start elasticsearch.

And then the logs starting from the first log line.

Hello Stephen,
I was following this dev.to/elastic/downloading-elasticsearch-and-kibana-macos-linux-and-windows-1mmo#for-windows

From the command line.

Thanks
-Karthik

@Karthikeyan_K You should follow our official docs, that blog is very old...

That would be my suggestion...

1 Like

Hello Stephen, Thanks for your response. I downloaded the latest version 8.11.4 for windows and followed the instructions. For some reasons I am getting the same error message. Anything to do with java jvm? Any other pointers pls.

"[2024-02-05T16:01:46,029][INFO ][o.a.l.i.v.PanamaVectorizationProvider] [DESKTOP-8JCMOL0] Java vector incubator API enabled; uses preferredBitSize=256
[2024-02-05T16:01:46,750][INFO ][o.e.n.Node               ] [DESKTOP-8JCMOL0] version[8.11.4], pid[2680], build[zip/da06c53fd49b7e676ccf8a32d6655c5155c16d81/2024-01-08T10:05:08.438562403Z], OS[Windows 11/10.0/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/21.0.1/21.0.1+12-29]
[2024-02-05T16:01:46,766][INFO ][o.e.n.Node               ] [DESKTOP-8JCMOL0] JVM home [C:\elastic_stack\elasticsearch-8.11.4-windows\elasticsearch-8.11.4\jdk], using bundled JDK [true]
[2024-02-05T16:01:46,767][INFO ][o.e.n.Node               ] [DESKTOP-8JCMOL0] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=org.elasticsearch.preallocate, -XX:+UseG1GC, -Djava.io.tmpdir=C:\Users\KARTHI~1\AppData\Local\Temp\elasticsearch, --add-modules=jdk.incubator.vector, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m, -Xms4049m, -Xmx4049m, -XX:MaxDirectMemorySize=2123366400, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.distribution.type=zip, --module-path=C:\elastic_stack\elasticsearch-8.11.4-windows\elasticsearch-8.11.4\lib, --add-modules=jdk.net, --add-modules=ALL-MODULE-PATH, -Djdk.module.main=org.elasticsearch.server]
[2024-02-05T16:01:47,033][ERROR][o.e.b.Elasticsearch      ] [DESKTOP-8JCMOL0] fatal exception while booting Elasticsearch
java.lang.IllegalStateException: failed to load plugin repository-url due to jar hell
	at org.elasticsearch.plugins.PluginsUtils.checkBundleJarHell(PluginsUtils.java:290) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:304) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:175) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.plugins.PluginsService.lambda$getPluginsServiceCtor$16(PluginsService.java:725) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.node.Node.<init>(Node.java:438) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.node.Node.<init>(Node.java:344) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:236) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:236) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:73) ~[elasticsearch-8.11.4.jar:?]
Caused by: java.lang.IllegalStateException: jar hell!
class: net.jpountz.lz4.LZ4BlockInputStream
jar1: C:\elastic_stack\elasticsearch-8.11.4-windows\elasticsearch-8.11.4\lib\lz4-java-1.8.0.jar
jar2: C:\PROGRA~1\IBM\SQLLIB\java\db2jcc4.jar
	at org.elasticsearch.jdk.JarHell.checkClass(JarHell.java:315) ~[elasticsearch-core-8.11.4.jar:?]
	at org.elasticsearch.jdk.JarHell.checkJarHell(JarHell.java:233) ~[elasticsearch-core-8.11.4.jar:?]
	at org.elasticsearch.plugins.PluginsUtils.checkBundleJarHell(PluginsUtils.java:288) ~[elasticsearch-8.11.4.jar:?]
	... 8 more
[2024-02-05T16:02:58,529][INFO ][o.a.l.i.v.PanamaVectorizationProvider] [DESKTOP-8JCMOL0] Java vector incubator API enabled; uses preferredBitSize=256
[2024-02-05T16:02:59,284][INFO ][o.e.n.Node               ] [DESKTOP-8JCMOL0] version[8.11.4], pid[5596], build[zip/da06c53fd49b7e676ccf8a32d6655c5155c16d81/2024-01-08T10:05:08.438562403Z], OS[Windows 11/10.0/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/21.0.1/21.0.1+12-29]
[2024-02-05T16:02:59,284][INFO ][o.e.n.Node               ] [DESKTOP-8JCMOL0] JVM home [C:\elastic_stack\elasticsearch-8.11.4-windows\elasticsearch-8.11.4\jdk], using bundled JDK [true]
[2024-02-05T16:02:59,299][INFO ][o.e.n.Node               ] [DESKTOP-8JCMOL0] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=org.elasticsearch.preallocate, -XX:+UseG1GC, -Djava.io.tmpdir=C:\Users\KARTHI~1\AppData\Local\Temp\elasticsearch, --add-modules=jdk.incubator.vector, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m, -Xms4049m, -Xmx4049m, -XX:MaxDirectMemorySize=2123366400, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.distribution.type=zip, --module-path=C:\elastic_stack\elasticsearch-8.11.4-windows\elasticsearch-8.11.4\lib, --add-modules=jdk.net, --add-modules=ALL-MODULE-PATH, -Djdk.module.main=org.elasticsearch.server]
[2024-02-05T16:02:59,566][ERROR][o.e.b.Elasticsearch      ] [DESKTOP-8JCMOL0] fatal exception while booting Elasticsearch
java.lang.IllegalStateException: failed to load plugin repository-url due to jar hell
	at org.elasticsearch.plugins.PluginsUtils.checkBundleJarHell(PluginsUtils.java:290) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:304) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:175) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.plugins.PluginsService.lambda$getPluginsServiceCtor$16(PluginsService.java:725) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.node.Node.<init>(Node.java:438) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.node.Node.<init>(Node.java:344) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:236) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:236) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:73) ~[elasticsearch-8.11.4.jar:?]
Caused by: java.lang.IllegalStateException: jar hell!
class: net.jpountz.lz4.LZ4BlockInputStream
jar1: C:\PROGRA~1\IBM\SQLLIB\java\db2jcc4.jar
jar2: C:\elastic_stack\elasticsearch-8.11.4-windows\elasticsearch-8.11.4\lib\lz4-java-1.8.0.jar
	at org.elasticsearch.jdk.JarHell.checkClass(JarHell.java:315) ~[elasticsearch-core-8.11.4.jar:?]
	at org.elasticsearch.jdk.JarHell.checkJarHell(JarHell.java:233) ~[elasticsearch-core-8.11.4.jar:?]
	at org.elasticsearch.plugins.PluginsUtils.checkBundleJarHell(PluginsUtils.java:288) ~[elasticsearch-8.11.4.jar:?]
	... 8 more
[2024-02-05T16:06:13,319][INFO ][o.a.l.i.v.PanamaVectorizationProvider] [DESKTOP-8JCMOL0] Java vector incubator API enabled; uses preferredBitSize=256
[2024-02-05T16:06:14,213][INFO ][o.e.n.Node               ] [DESKTOP-8JCMOL0] version[8.11.4], pid[20536], build[zip/da06c53fd49b7e676ccf8a32d6655c5155c16d81/2024-01-08T10:05:08.438562403Z], OS[Windows 11/10.0/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/21.0.1/21.0.1+12-29]
[2024-02-05T16:06:14,214][INFO ][o.e.n.Node               ] [DESKTOP-8JCMOL0] JVM home [C:\elastic_stack\elasticsearch-8.11.4-windows\elasticsearch-8.11.4\jdk], using bundled JDK [true]
[2024-02-05T16:06:14,215][INFO ][o.e.n.Node               ] [DESKTOP-8JCMOL0] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=org.elasticsearch.preallocate, -XX:+UseG1GC, -Djava.io.tmpdir=C:\Users\KARTHI~1\AppData\Local\Temp\elasticsearch, --add-modules=jdk.incubator.vector, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m, -Xms4049m, -Xmx4049m, -XX:MaxDirectMemorySize=2123366400, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, -Des.distribution.type=zip, --module-path=C:\elastic_stack\elasticsearch-8.11.4-windows\elasticsearch-8.11.4\lib, --add-modules=jdk.net, --add-modules=ALL-MODULE-PATH, -Djdk.module.main=org.elasticsearch.server]
[2024-02-05T16:06:14,505][ERROR][o.e.b.Elasticsearch      ] [DESKTOP-8JCMOL0] fatal exception while booting Elasticsearch
java.lang.IllegalStateException: failed to load plugin repository-url due to jar hell
	at org.elasticsearch.plugins.PluginsUtils.checkBundleJarHell(PluginsUtils.java:290) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:304) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:175) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.plugins.PluginsService.lambda$getPluginsServiceCtor$16(PluginsService.java:725) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.node.Node.<init>(Node.java:438) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.node.Node.<init>(Node.java:344) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.bootstrap.Elasticsearch$2.<init>(Elasticsearch.java:236) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:236) ~[elasticsearch-8.11.4.jar:?]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:73) ~[elasticsearch-8.11.4.jar:?]
Caused by: java.lang.IllegalStateException: jar hell!
class: net.jpountz.lz4.LZ4BlockInputStream
jar1: C:\PROGRA~1\IBM\SQLLIB\java\db2jcc4.jar
jar2: C:\elastic_stack\elasticsearch-8.11.4-windows\elasticsearch-8.11.4\lib\lz4-java-1.8.0.jar
	at org.elasticsearch.jdk.JarHell.checkClass(JarHell.java:315) ~[elasticsearch-core-8.11.4.jar:?]
	at org.elasticsearch.jdk.JarHell.checkJarHell(JarHell.java:233) ~[elasticsearch-core-8.11.4.jar:?]
	at org.elasticsearch.plugins.PluginsUtils.checkBundleJarHell(PluginsUtils.java:288) ~[elasticsearch-8.11.4.jar:?]
	... 8 more

Exactly what command did you start Elasticsearch with?

Exactly. How did you install?

Please show the directory and the exact command you ran and the log starting from that command.

You can see you have two competing JVMs.

When installed correctly and ran correctly, elasticsearch should use its own bundled JVM

Adding to @stephenb's answer....

What is this? Did you change the classpath or the jvm options or anything like that?

jar2: C:\PROGRA~1\IBM\SQLLIB\java\db2jcc4.jar

I placed the downloaded zip in

` ```C:\elastic_stack then navigated to C:\elastic_stack\elasticsearch-8.11.4-windows\elasticsearch-8.11.4\bin using command prompt in admin mode. Then ran the Elasticsearch.bat batch file.
C:\Windows\System32>cd C:\elastic_stack\elasticsearch-8.11.4-windows\elasticsearch-8.11.4\bin

C:\elastic_stack\elasticsearch-8.11.4-windows\elasticsearch-8.11.4\bin>elasticsearch.bat ```

Probably it is picking from prev application install. I did not change any settings. Should I remove anything from the Env variable?

What do you have as "env variables"?

CLASSPATH has set of URLS all pointing to IBM folder. Let me know if anything has to be changed.

This is not something set by elastic.

Ok super important

Do not navigate all the way to the bin directory.

Do not run with a full path.

There are very specific instructions on how to run from the command line.

Navigate to the elastic home directory

C:\elastic_stack\elasticsearch-8.11.4-windows\elasticsearch-8.11.4\

Then run

.\bin\elasticsearch.bat

This is the proper way and what the document shows

You need to run from the home directory because then relative paths from there are correct

Correct. This is set by some other app install (DB 2 client I think). Is this affecting Elastic deployment?

Ok. Tried that Stephen. Same results. Should I try uninstalling the other IBM app and check?

Hmmm looks like there is a conflict... then So yes uninstall the other app or try elasticsearch on another server or container