# Elasticsearch service not starting

**URL:** <https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534>\
**Category:** Elasticsearch\
**Created:** [January 10, 2020, 5:42am UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534 "2020-01-10T05:42:16Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![gsumilhig](https://avatars.discourse-cdn.com/v4/letter/g/e9bcb4/32.png) [@gsumilhig](https://discuss.elastic.co/u/gsumilhig)\
**Post date:** [January 10, 2020, 5:42am UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534/1 "2020-01-10T05:42:16Z")

</div>

Caused by: java.lang.IllegalStateException: jar hell!  
class: org.elasticsearch.plugins.ExtendedPluginsClassLoader  
jar1: /usr/share/elasticsearch/lib/plugin-classloader-6.8.0.jar  
jar2: /usr/share/elasticsearch/lib/plugin-classloader-6.8.3.jar  
at org.elasticsearch.bootstrap.JarHell.checkClass(JarHell.java:277) ~[elasticsearch-core-6.8.3.jar:6.8.3]  
at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:190) ~[elasticsearch-core-6.8.3.jar:6.8.3]  
at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:86) ~[elasticsearch-core-6.8.3.jar:6.8.3]  
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:197) ~[elasticsearch-6.8.3.jar:6.8.3]  
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.8.3.jar:6.8.3]  
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.8.3.jar:6.8.3]  
... 6 more

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 10, 2020, 5:59am UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534/2 "2020-01-10T05:59:36Z")

</div>

Welcome!

It could help if you explain what you did exactly to get in that situation.  
Here it seems that 2 versions are mixed together.

---

<div class="post-metadata">

**Author:** ![gsumilhig](https://avatars.discourse-cdn.com/v4/letter/g/e9bcb4/32.png) [@gsumilhig](https://discuss.elastic.co/u/gsumilhig)\
**Post date:** [January 10, 2020, 6:16am UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534/3 "2020-01-10T06:16:56Z")

</div>

Hi,

We just shutdown the VM and increase the disk because /var/log/elasticsearh is 100% full.. after that we found that ther service didn't start.

# systemctl status elasticsearch

● elasticsearch.service - Elasticsearch  
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)  
Active: failed (Result: exit-code) since Fri 2020-01-10 13:51:50 PST; 24min ago  
Docs: [http://www.elastic.co](http://www.elastic.co)  
Process: 4637 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID\_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)  
Main PID: 4637 (code=exited, status=1/FAILURE)

Jan 10 13:51:47 elastic01 systemd[1]: Started Elasticsearch.  
Jan 10 13:51:50 elastic01 systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE  
Jan 10 13:51:50 elastic01 systemd[1]: Unit elasticsearch.service entered failed state.  
Jan 10 13:51:50 elastic01 systemd[1]: elasticsearch.service failed.

---

<div class="post-metadata">

**Author:** ![gsumilhig](https://avatars.discourse-cdn.com/v4/letter/g/e9bcb4/32.png) [@gsumilhig](https://discuss.elastic.co/u/gsumilhig)\
**Post date:** [January 10, 2020, 6:19am UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534/4 "2020-01-10T06:19:25Z")

</div>

We have 3 nodes and we applied the disk increase in the first node only.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 10, 2020, 6:35am UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534/5 "2020-01-10T06:35:46Z")

</div>

Pretty sure that something else happened.  
Like an upgrade which was wrongly executed.

---

<div class="post-metadata">

**Author:** ![gsumilhig](https://avatars.discourse-cdn.com/v4/letter/g/e9bcb4/32.png) [@gsumilhig](https://discuss.elastic.co/u/gsumilhig)\
**Post date:** [January 10, 2020, 7:06am UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534/6 "2020-01-10T07:06:29Z")

</div>

Hi,

Is it possible to delete some files in cluster /var/log/elasticsearch?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 10, 2020, 7:37am UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534/7 "2020-01-10T07:37:25Z")

</div>

The trouble seems to be in the `plugins` dir here.  
What do you have inside?

---

<div class="post-metadata">

**Author:** ![gsumilhig](https://avatars.discourse-cdn.com/v4/letter/g/e9bcb4/32.png) [@gsumilhig](https://discuss.elastic.co/u/gsumilhig)\
**Post date:** [January 10, 2020, 7:41am UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534/8 "2020-01-10T07:41:18Z")

</div>

there's no file inside /usr/share/elasticsearch/plugins. same with the other nodes

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 10, 2020, 7:59am UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534/9 "2020-01-10T07:59:36Z")

</div>

Could you share the full elasticsearch logs please?

Please format your logs. Paste the text and format it with \</\> icon or pairs of triple backticks (```), and check the preview window to make sure it's properly formatted before posting it. This makes it more likely that your question will receive a useful answer.

---

<div class="post-metadata">

**Author:** ![gsumilhig](https://avatars.discourse-cdn.com/v4/letter/g/e9bcb4/32.png) [@gsumilhig](https://discuss.elastic.co/u/gsumilhig)\
**Post date:** [January 10, 2020, 9:14am UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534/10 "2020-01-10T09:14:42Z")

</div>

````auto
java.lang.IllegalStateException: jar hell!
class: org.elasticsearch.plugins.ExtendedPluginsClassLoader
jar1: /usr/share/elasticsearch/lib/plugin-classloader-6.8.0.jar
jar2: /usr/share/elasticsearch/lib/plugin-classloader-6.8.3.jar
        at org.elasticsearch.bootstrap.JarHell.checkClass(JarHell.java:277) ~[elasticsearch-core-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:190) ~[elasticsearch-core-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:86) ~[elasticsearch-core-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:197) ~[elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.8.0.jar:6.8.3]
        at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.8.0.jar:6.8.3]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) [elasticsearch-6.8.3.jar:6.8.3]
[2020-01-10T14:52:27,833][WARN][o.e.b.ElasticsearchUncaughtExceptionHandler] [elastic01] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: jar hell!
class: org.elasticsearch.plugins.ExtendedPluginsClassLoader
jar1: /usr/share/elasticsearch/lib/plugin-classloader-6.8.0.jar
jar2: /usr/share/elasticsearch/lib/plugin-classloader-6.8.3.jar
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.8.0.jar:6.8.3]
        at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.8.0.jar:6.8.3]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) ~[elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.8.3.jar:6.8.3]
Caused by: java.lang.IllegalStateException: jar hell!
class: org.elasticsearch.plugins.ExtendedPluginsClassLoader
jar1: /usr/share/elasticsearch/lib/plugin-classloader-6.8.0.jar
jar2: /usr/share/elasticsearch/lib/plugin-classloader-6.8.3.jar
        at org.elasticsearch.bootstrap.JarHell.checkClass(JarHell.java:277) ~[elasticsearch-core-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:190) ~[elasticsearch-core-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:86) ~[elasticsearch-core-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:197) ~[elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.8.3.jar:6.8.3]
        ... 6 more```

Thank you.
````

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 10, 2020, 9:57am UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534/11 "2020-01-10T09:57:18Z")

</div>

I see.

Could list the files which are in the elasticsearch `lib` directory?

---

<div class="post-metadata">

**Author:** ![gsumilhig](https://avatars.discourse-cdn.com/v4/letter/g/e9bcb4/32.png) [@gsumilhig](https://discuss.elastic.co/u/gsumilhig)\
**Post date:** [January 13, 2020, 1:46am UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534/12 "2020-01-13T01:46:58Z")

</div>

# ls /usr/share/elasticsearch/lib/

elasticsearch-6.8.0.jar hppc-0.7.1.jar log4j-api-2.11.1.jar lucene-sandbox-7.7.0.jar  
elasticsearch-6.8.3.jar jackson-core-2.8.11.jar log4j-core-2.11.1.jar lucene-spatial3d-7.7.0.jar  
elasticsearch-cli-6.8.0.jar jackson-dataformat-cbor-2.8.11.jar lucene-analyzers-common-7.7.0.jar lucene-spatial-7.7.0.jar  
elasticsearch-core-6.8.0.jar jackson-dataformat-smile-2.8.11.jar lucene-backward-codecs-7.7.0.jar lucene-spatial-extras-7.7.0.jar  
elasticsearch-core-6.8.3.jar jackson-dataformat-yaml-2.8.11.jar lucene-core-7.7.0.jar lucene-suggest-7.7.0.jar  
elasticsearch-launchers-6.8.0.jar java-version-checker-6.8.0.jar lucene-grouping-7.7.0.jar plugin-classloader-6.8.0.jar  
elasticsearch-launchers-6.8.3.jar java-version-checker-6.8.3.jar lucene-highlighter-7.7.0.jar plugin-classloader-6.8.3.jar  
elasticsearch-secure-sm-6.8.0.jar jna-4.5.1.jar lucene-join-7.7.0.jar snakeyaml-1.17.jar  
elasticsearch-secure-sm-6.8.3.jar joda-time-2.10.1.jar lucene-memory-7.7.0.jar spatial4j-0.7.jar  
elasticsearch-x-content-6.8.0.jar jopt-simple-5.0.2.jar lucene-misc-7.7.0.jar t-digest-3.2.jar  
elasticsearch-x-content-6.8.3.jar jts-core-1.15.0.jar lucene-queries-7.7.0.jar tools  
HdrHistogram-2.1.9.jar log4j-1.2-api-2.11.1.jar lucene-queryparser-7.7.0.jar

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 13, 2020, 5:05am UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534/13 "2020-01-13T05:05:42Z")

</div>

You have a mix of 2 elasticsearch versions.  
You should reinstall correctly elasticsearch but keep the `data` dir and the `config` dir.

---

<div class="post-metadata">

**Author:** ![gsumilhig](https://avatars.discourse-cdn.com/v4/letter/g/e9bcb4/32.png) [@gsumilhig](https://discuss.elastic.co/u/gsumilhig)\
**Post date:** [January 13, 2020, 6:02am UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534/14 "2020-01-13T06:02:19Z")

</div>

Error after reinstall elasticsearch using same version.

````auto
[2020-01-13T13:38:50,082][WARN][o.e.b.Natives] [elastic01] unable to load JNA native support library, native methods will be disabled.
java.lang.UnsatisfiedLinkError: /tmp/elasticsearch-8296603538933134483/jna--1985354563/jna6449887839057156505.tmp: /tmp/elasticsearch-8296603538933134483/jna--1985354563/jna6449887839057156505.tmp: failed to map segment from shared object: Operation not permitted
        at java.lang.ClassLoader$NativeLibrary.load(Native Method) ~[?:1.8.0_222]
        at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) ~[?:1.8.0_222]
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824) ~[?:1.8.0_222]
        at java.lang.Runtime.load0(Runtime.java:809) ~[?:1.8.0_222]
        at java.lang.System.load(System.java:1086) ~[?:1.8.0_222]
        at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:947) ~[jna-4.5.1.jar:4.5.1 (b0)]
        at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:922) ~[jna-4.5.1.jar:4.5.1 (b0)]
        at com.sun.jna.Native.<clinit>(Native.java:190) ~[jna-4.5.1.jar:4.5.1 (b0)]
        at java.lang.Class.forName0(Native Method) ~[?:1.8.0_222]
        at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_222]
        at org.elasticsearch.bootstrap.Natives.<clinit>(Natives.java:45) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:102) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.8.0.jar:6.8.3]
        at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.8.0.jar:6.8.3]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) [elasticsearch-6.8.3.jar:6.8.3]
[2020-01-13T13:38:50,089][WARN][o.e.b.Natives] [elastic01] cannot check if running as root because JNA is not available
[2020-01-13T13:38:50,089][WARN][o.e.b.Natives] [elastic01] cannot register console handler because JNA is not available
[2020-01-13T13:38:50,090][WARN][o.e.b.Natives] [elastic01] cannot getrlimit RLIMIT_NPROC because JNA is not available
[2020-01-13T13:38:50,090][WARN][o.e.b.Natives] [elastic01] cannot getrlimit RLIMIT_AS because JNA is not available
[2020-01-13T13:38:50,091][WARN][o.e.b.Natives] [elastic01] cannot getrlimit RLIMIT_FSIZE because JNA is not available
[2020-01-13T13:38:50,125][ERROR][o.e.b.Bootstrap] [elastic01] Exception
java.lang.IllegalStateException: jar hell!
class: META-INF.versions.9.org.elasticsearch.monitor.jvm.JvmPid
jar1: /usr/share/elasticsearch/lib/elasticsearch-6.8.3.jar
jar2: /usr/share/elasticsearch/lib/elasticsearch-6.8.0.jar
        at org.elasticsearch.bootstrap.JarHell.checkClass(JarHell.java:277) ~[elasticsearch-core-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:190) ~[elasticsearch-core-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:86) ~[elasticsearch-core-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:197) ~[elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.8.0.jar:6.8.3]
        at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.8.0.jar:6.8.3]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) [elasticsearch-6.8.3.jar:6.8.3]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) [elasticsearch-6.8.3.jar:6.8.3]```
````

---

<div class="post-metadata">

**Author:** ![gsumilhig](https://avatars.discourse-cdn.com/v4/letter/g/e9bcb4/32.png) [@gsumilhig](https://discuss.elastic.co/u/gsumilhig)\
**Post date:** [January 13, 2020, 7:43am UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534/15 "2020-01-13T07:43:02Z")

</div>

it works now, by uninstalling elasticsearch and remove some elasticsearch file lib..

Thank you dadoonet

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 10, 2020, 7:43am UTC](https://discuss.elastic.co/t/elasticsearch-service-not-starting/214534/16 "2020-02-10T07:43:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
