Fatal exception while booting Elasticsearch

Abhi_Abhishek · October 18, 2022, 12:10pm

Hello,
While I am trying to upgrade from 8.3.2 to 8.4.1 Its upgrading the Elasticsearch but the service is not running.

[2022-10-18T11:15:37,060][ERROR][o.e.b.Elasticsearch      ] [vm0dnelkdma0003.corp.chartercom.com] fatal exception while booting Elasticsearch
java.lang.ExceptionInInitializerError: null
        at org.elasticsearch.bootstrap.JNANatives.definitelyRunningAsRoot(JNANatives.java:162) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Natives.definitelyRunningAsRoot(Natives.java:57) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initializeNatives(Elasticsearch.java:259) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initPhase2(Elasticsearch.java:166) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:66) ~[elasticsearch-8.4.1.jar:?]
Caused by: java.lang.UnsupportedOperationException: Failed to allocate closure
        at com.sun.jna.Native.registerMethod(Native Method) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1906) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1775) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1493) ~[jna-5.10.0.jar:?]
        at org.elasticsearch.bootstrap.JNACLibrary.<clinit>(JNACLibrary.java:38) ~[elasticsearch-8.4.1.jar:?]

warkolm · October 19, 2022, 3:13am

How did you upgrade?
Can you share more of the log please.

Abhi_Abhishek · October 19, 2022, 3:45am

I have upgraded it by RPM package by using the command sudo rpm --upgrade elasticsearch...
The logs are only this much it was reapeating whenever I try to start a elastic service.

[2022-10-18T06:06:15,371][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [vm0dnelkdma0003.corp.chartercom.com] evicted [0] entries from cache after reloading database [/tmp/elasticsearch-16319186703517499351/geoip-databases/Wh7QmrtaQs20kcOkcYtUTw/GeoLite2-ASN.mmdb]
[2022-10-18T06:06:15,382][INFO ][o.e.i.g.DatabaseNodeService] [vm0dnelkdma0003.corp.chartercom.com] successfully loaded geoip database file [GeoLite2-ASN.mmdb]
[2022-10-18T06:06:17,913][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [vm0dnelkdma0003.corp.chartercom.com] evicted [0] entries from cache after reloading database [/tmp/elasticsearch-16319186703517499351/geoip-databases/Wh7QmrtaQs20kcOkcYtUTw/GeoLite2-Country.mmdb]
[2022-10-18T06:06:17,913][INFO ][o.e.i.g.DatabaseNodeService] [vm0dnelkdma0003.corp.chartercom.com] successfully loaded geoip database file [GeoLite2-Country.mmdb]
[2022-10-18T06:06:26,655][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [vm0dnelkdma0003.corp.chartercom.com] evicted [0] entries from cache after reloading database [/tmp/elasticsearch-16319186703517499351/geoip-databases/Wh7QmrtaQs20kcOkcYtUTw/GeoLite2-City.mmdb]
[2022-10-18T06:06:26,656][INFO ][o.e.i.g.DatabaseNodeService] [vm0dnelkdma0003.corp.chartercom.com] successfully loaded geoip database file [GeoLite2-City.mmdb]
[2022-10-18T10:55:21,712][INFO ][o.e.t.ClusterConnectionManager] [vm0dnelkdma0003.corp.chartercom.com] transport connection to [{vm0dnelkdma0001.corp.chartercom.com}{X6EckQreTvmJEDlWCcn8CA}{WPLd1lT-RVu3K891dIM7tw}{vm0dnelkdma0001.corp.chartercom.com}{vm0dnelkdma0001.corp.chartercom.com}{22.240.82.10:9300}{ir}] closed by remote
[2022-10-18T10:55:21,728][INFO ][o.e.c.s.ClusterApplierService] [vm0dnelkdma0003.corp.chartercom.com] removed {{vm0dnelkdma0001.corp.chartercom.com}{X6EckQreTvmJEDlWCcn8CA}{WPLd1lT-RVu3K891dIM7tw}{vm0dnelkdma0001.corp.chartercom.com}{vm0dnelkdma0001.corp.chartercom.com}{22.240.82.10:9300}{ir}}, term: 131, version: 72021, reason: ApplyCommitRequest{term=131, version=72021, sourceNode={vm0dnelkdma0002.corp.chartercom.com}{Z9XzmY8YTy6Vj_tTNF7tpA}{SZ8Gl_6aQb63yasjx88Sbw}{vm0dnelkdma0002.corp.chartercom.com}{vm0dnelkdma0002.corp.chartercom.com}{22.240.82.12:9300}{cdfhilmrstw}{ml.machine_memory=67387314176, ml.max_jvm_size=33285996544, xpack.installed=true}}
[2022-10-18T11:11:12,459][INFO ][o.e.n.Node               ] [vm0dnelkdma0003.corp.chartercom.com] stopping ...
[2022-10-18T11:11:12,463][INFO ][o.e.x.w.WatcherService   ] [vm0dnelkdma0003.corp.chartercom.com] stopping watch service, reason [shutdown initiated]
[2022-10-18T11:11:12,464][INFO ][o.e.x.w.WatcherLifeCycleService] [vm0dnelkdma0003.corp.chartercom.com] watcher has stopped and shutdown
[2022-10-18T11:11:12,780][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [vm0dnelkdma0003.corp.chartercom.com] [controller/8876] [Main.cc@176] ML controller exiting
[2022-10-18T11:11:12,781][INFO ][o.e.x.m.p.NativeController] [vm0dnelkdma0003.corp.chartercom.com] Native controller process has stopped - no new native processes can be started
[2022-10-18T11:11:12,780][INFO ][o.e.c.c.Coordinator      ] [vm0dnelkdma0003.corp.chartercom.com] master node [{vm0dnelkdma0002.corp.chartercom.com}{Z9XzmY8YTy6Vj_tTNF7tpA}{SZ8Gl_6aQb63yasjx88Sbw}{vm0dnelkdma0002.corp.chartercom.com}{vm0dnelkdma0002.corp.chartercom.com}{22.240.82.12:9300}{cdfhilmrstw}] disconnected, restarting discovery
[2022-10-18T11:11:15,893][INFO ][o.e.n.Node               ] [vm0dnelkdma0003.corp.chartercom.com] stopped
[2022-10-18T11:11:15,893][INFO ][o.e.n.Node               ] [vm0dnelkdma0003.corp.chartercom.com] closing ...
[2022-10-18T11:11:15,901][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [vm0dnelkdma0003.corp.chartercom.com] evicted [0] entries from cache after reloading database [/tmp/elasticsearch-16319186703517499351/geoip-databases/Wh7QmrtaQs20kcOkcYtUTw/GeoLite2-Country.mmdb]
[2022-10-18T11:11:15,901][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [vm0dnelkdma0003.corp.chartercom.com] evicted [0] entries from cache after reloading database [/tmp/elasticsearch-16319186703517499351/geoip-databases/Wh7QmrtaQs20kcOkcYtUTw/GeoLite2-ASN.mmdb]
[2022-10-18T11:11:15,901][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [vm0dnelkdma0003.corp.chartercom.com] evicted [0] entries from cache after reloading database [/tmp/elasticsearch-16319186703517499351/geoip-databases/Wh7QmrtaQs20kcOkcYtUTw/GeoLite2-City.mmdb]
[2022-10-18T11:11:15,941][INFO ][o.e.n.Node               ] [vm0dnelkdma0003.corp.chartercom.com] closed
[2022-10-18T11:15:37,060][ERROR][o.e.b.Elasticsearch      ] [vm0dnelkdma0003.corp.chartercom.com] fatal exception while booting Elasticsearch
java.lang.ExceptionInInitializerError: null
        at org.elasticsearch.bootstrap.JNANatives.definitelyRunningAsRoot(JNANatives.java:162) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Natives.definitelyRunningAsRoot(Natives.java:57) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initializeNatives(Elasticsearch.java:259) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initPhase2(Elasticsearch.java:166) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:66) ~[elasticsearch-8.4.1.jar:?]
Caused by: java.lang.UnsupportedOperationException: Failed to allocate closure
        at com.sun.jna.Native.registerMethod(Native Method) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1906) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1775) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1493) ~[jna-5.10.0.jar:?]
        at org.elasticsearch.bootstrap.JNACLibrary.<clinit>(JNACLibrary.java:38) ~[elasticsearch-8.4.1.jar:?]
        ... 5 more
[2022-10-18T11:29:29,417][ERROR][o.e.b.Elasticsearch      ] [vm0dnelkdma0003.corp.chartercom.com] fatal exception while booting Elasticsearch
java.lang.ExceptionInInitializerError: null
        at org.elasticsearch.bootstrap.JNANatives.definitelyRunningAsRoot(JNANatives.java:162) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Natives.definitelyRunningAsRoot(Natives.java:57) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initializeNatives(Elasticsearch.java:259) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initPhase2(Elasticsearch.java:166) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:66) ~[elasticsearch-8.4.1.jar:?]
Caused by: java.lang.UnsupportedOperationException: Failed to allocate closure
        at com.sun.jna.Native.registerMethod(Native Method) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1906) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1775) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1493) ~[jna-5.10.0.jar:?]
        at org.elasticsearch.bootstrap.JNACLibrary.<clinit>(JNACLibrary.java:38) ~[elasticsearch-8.4.1.jar:?]
        ... 5 more
[2022-10-18T12:27:08,433][ERROR][o.e.b.Elasticsearch      ] [vm0dnelkdma0003.corp.chartercom.com] fatal exception while booting Elasticsearch
java.lang.ExceptionInInitializerError: null
        at org.elasticsearch.bootstrap.JNANatives.definitelyRunningAsRoot(JNANatives.java:162) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Natives.definitelyRunningAsRoot(Natives.java:57) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initializeNatives(Elasticsearch.java:259) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initPhase2(Elasticsearch.java:166) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:66) ~[elasticsearch-8.4.1.jar:?]
Caused by: java.lang.UnsupportedOperationException: Failed to allocate closure
        at com.sun.jna.Native.registerMethod(Native Method) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1906) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1775) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1493) ~[jna-5.10.0.jar:?]
        at org.elasticsearch.bootstrap.JNACLibrary.<clinit>(JNACLibrary.java:38) ~[elasticsearch-8.4.1.jar:?]
        ... 5 more

warkolm · October 19, 2022, 3:59am

This suggests that something has asked Elasticsearch to shutdown.

I would look in your OS logs to see if there is anything related.

system · November 16, 2022, 4:00am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
We are seeing the issue on SonarQube with elasticsearch. Elastic search is not coming up preventing the sonarqube to be up and running Elasticsearch	2	937	August 22, 2023
Getting fatal exception while booting Elasticsearch while trying to start elasticsearch in Vm Elasticsearch migration	4	499	February 14, 2024
Unable to start Elasticsearch 7.6.2 Elasticsearch	1	604	December 1, 2020
Just downloaded elastic search version 8.13.2 and ran the elasticsearch.bat. it return with this error. Elasticsearch died while starting up, with exit code 1 Elasticsearch	10	893	May 18, 2024
Java Transport Client 5.4.1 NullPointerException Elasticsearch	16	2595	July 19, 2017

Fatal exception while booting Elasticsearch

Related topics