Fatal exception while booting Elasticsearch

Hello,
While I am trying to upgrade from 8.3.2 to 8.4.1 Its upgrading the Elasticsearch but the service is not running.

[2022-10-18T11:15:37,060][ERROR][o.e.b.Elasticsearch      ] [vm0dnelkdma0003.corp.chartercom.com] fatal exception while booting Elasticsearch
java.lang.ExceptionInInitializerError: null
        at org.elasticsearch.bootstrap.JNANatives.definitelyRunningAsRoot(JNANatives.java:162) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Natives.definitelyRunningAsRoot(Natives.java:57) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initializeNatives(Elasticsearch.java:259) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initPhase2(Elasticsearch.java:166) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:66) ~[elasticsearch-8.4.1.jar:?]
Caused by: java.lang.UnsupportedOperationException: Failed to allocate closure
        at com.sun.jna.Native.registerMethod(Native Method) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1906) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1775) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1493) ~[jna-5.10.0.jar:?]
        at org.elasticsearch.bootstrap.JNACLibrary.<clinit>(JNACLibrary.java:38) ~[elasticsearch-8.4.1.jar:?]

How did you upgrade?
Can you share more of the log please.

I have upgraded it by RPM package by using the command sudo rpm --upgrade elasticsearch...
The logs are only this much it was reapeating whenever I try to start a elastic service.

[2022-10-18T06:06:15,371][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [vm0dnelkdma0003.corp.chartercom.com] evicted [0] entries from cache after reloading database [/tmp/elasticsearch-16319186703517499351/geoip-databases/Wh7QmrtaQs20kcOkcYtUTw/GeoLite2-ASN.mmdb]
[2022-10-18T06:06:15,382][INFO ][o.e.i.g.DatabaseNodeService] [vm0dnelkdma0003.corp.chartercom.com] successfully loaded geoip database file [GeoLite2-ASN.mmdb]
[2022-10-18T06:06:17,913][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [vm0dnelkdma0003.corp.chartercom.com] evicted [0] entries from cache after reloading database [/tmp/elasticsearch-16319186703517499351/geoip-databases/Wh7QmrtaQs20kcOkcYtUTw/GeoLite2-Country.mmdb]
[2022-10-18T06:06:17,913][INFO ][o.e.i.g.DatabaseNodeService] [vm0dnelkdma0003.corp.chartercom.com] successfully loaded geoip database file [GeoLite2-Country.mmdb]
[2022-10-18T06:06:26,655][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [vm0dnelkdma0003.corp.chartercom.com] evicted [0] entries from cache after reloading database [/tmp/elasticsearch-16319186703517499351/geoip-databases/Wh7QmrtaQs20kcOkcYtUTw/GeoLite2-City.mmdb]
[2022-10-18T06:06:26,656][INFO ][o.e.i.g.DatabaseNodeService] [vm0dnelkdma0003.corp.chartercom.com] successfully loaded geoip database file [GeoLite2-City.mmdb]
[2022-10-18T10:55:21,712][INFO ][o.e.t.ClusterConnectionManager] [vm0dnelkdma0003.corp.chartercom.com] transport connection to [{vm0dnelkdma0001.corp.chartercom.com}{X6EckQreTvmJEDlWCcn8CA}{WPLd1lT-RVu3K891dIM7tw}{vm0dnelkdma0001.corp.chartercom.com}{vm0dnelkdma0001.corp.chartercom.com}{22.240.82.10:9300}{ir}] closed by remote
[2022-10-18T10:55:21,728][INFO ][o.e.c.s.ClusterApplierService] [vm0dnelkdma0003.corp.chartercom.com] removed {{vm0dnelkdma0001.corp.chartercom.com}{X6EckQreTvmJEDlWCcn8CA}{WPLd1lT-RVu3K891dIM7tw}{vm0dnelkdma0001.corp.chartercom.com}{vm0dnelkdma0001.corp.chartercom.com}{22.240.82.10:9300}{ir}}, term: 131, version: 72021, reason: ApplyCommitRequest{term=131, version=72021, sourceNode={vm0dnelkdma0002.corp.chartercom.com}{Z9XzmY8YTy6Vj_tTNF7tpA}{SZ8Gl_6aQb63yasjx88Sbw}{vm0dnelkdma0002.corp.chartercom.com}{vm0dnelkdma0002.corp.chartercom.com}{22.240.82.12:9300}{cdfhilmrstw}{ml.machine_memory=67387314176, ml.max_jvm_size=33285996544, xpack.installed=true}}
[2022-10-18T11:11:12,459][INFO ][o.e.n.Node               ] [vm0dnelkdma0003.corp.chartercom.com] stopping ...
[2022-10-18T11:11:12,463][INFO ][o.e.x.w.WatcherService   ] [vm0dnelkdma0003.corp.chartercom.com] stopping watch service, reason [shutdown initiated]
[2022-10-18T11:11:12,464][INFO ][o.e.x.w.WatcherLifeCycleService] [vm0dnelkdma0003.corp.chartercom.com] watcher has stopped and shutdown
[2022-10-18T11:11:12,780][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [vm0dnelkdma0003.corp.chartercom.com] [controller/8876] [Main.cc@176] ML controller exiting
[2022-10-18T11:11:12,781][INFO ][o.e.x.m.p.NativeController] [vm0dnelkdma0003.corp.chartercom.com] Native controller process has stopped - no new native processes can be started
[2022-10-18T11:11:12,780][INFO ][o.e.c.c.Coordinator      ] [vm0dnelkdma0003.corp.chartercom.com] master node [{vm0dnelkdma0002.corp.chartercom.com}{Z9XzmY8YTy6Vj_tTNF7tpA}{SZ8Gl_6aQb63yasjx88Sbw}{vm0dnelkdma0002.corp.chartercom.com}{vm0dnelkdma0002.corp.chartercom.com}{22.240.82.12:9300}{cdfhilmrstw}] disconnected, restarting discovery
[2022-10-18T11:11:15,893][INFO ][o.e.n.Node               ] [vm0dnelkdma0003.corp.chartercom.com] stopped
[2022-10-18T11:11:15,893][INFO ][o.e.n.Node               ] [vm0dnelkdma0003.corp.chartercom.com] closing ...
[2022-10-18T11:11:15,901][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [vm0dnelkdma0003.corp.chartercom.com] evicted [0] entries from cache after reloading database [/tmp/elasticsearch-16319186703517499351/geoip-databases/Wh7QmrtaQs20kcOkcYtUTw/GeoLite2-Country.mmdb]
[2022-10-18T11:11:15,901][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [vm0dnelkdma0003.corp.chartercom.com] evicted [0] entries from cache after reloading database [/tmp/elasticsearch-16319186703517499351/geoip-databases/Wh7QmrtaQs20kcOkcYtUTw/GeoLite2-ASN.mmdb]
[2022-10-18T11:11:15,901][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [vm0dnelkdma0003.corp.chartercom.com] evicted [0] entries from cache after reloading database [/tmp/elasticsearch-16319186703517499351/geoip-databases/Wh7QmrtaQs20kcOkcYtUTw/GeoLite2-City.mmdb]
[2022-10-18T11:11:15,941][INFO ][o.e.n.Node               ] [vm0dnelkdma0003.corp.chartercom.com] closed
[2022-10-18T11:15:37,060][ERROR][o.e.b.Elasticsearch      ] [vm0dnelkdma0003.corp.chartercom.com] fatal exception while booting Elasticsearch
java.lang.ExceptionInInitializerError: null
        at org.elasticsearch.bootstrap.JNANatives.definitelyRunningAsRoot(JNANatives.java:162) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Natives.definitelyRunningAsRoot(Natives.java:57) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initializeNatives(Elasticsearch.java:259) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initPhase2(Elasticsearch.java:166) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:66) ~[elasticsearch-8.4.1.jar:?]
Caused by: java.lang.UnsupportedOperationException: Failed to allocate closure
        at com.sun.jna.Native.registerMethod(Native Method) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1906) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1775) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1493) ~[jna-5.10.0.jar:?]
        at org.elasticsearch.bootstrap.JNACLibrary.<clinit>(JNACLibrary.java:38) ~[elasticsearch-8.4.1.jar:?]
        ... 5 more
[2022-10-18T11:29:29,417][ERROR][o.e.b.Elasticsearch      ] [vm0dnelkdma0003.corp.chartercom.com] fatal exception while booting Elasticsearch
java.lang.ExceptionInInitializerError: null
        at org.elasticsearch.bootstrap.JNANatives.definitelyRunningAsRoot(JNANatives.java:162) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Natives.definitelyRunningAsRoot(Natives.java:57) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initializeNatives(Elasticsearch.java:259) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initPhase2(Elasticsearch.java:166) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:66) ~[elasticsearch-8.4.1.jar:?]
Caused by: java.lang.UnsupportedOperationException: Failed to allocate closure
        at com.sun.jna.Native.registerMethod(Native Method) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1906) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1775) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1493) ~[jna-5.10.0.jar:?]
        at org.elasticsearch.bootstrap.JNACLibrary.<clinit>(JNACLibrary.java:38) ~[elasticsearch-8.4.1.jar:?]
        ... 5 more
[2022-10-18T12:27:08,433][ERROR][o.e.b.Elasticsearch      ] [vm0dnelkdma0003.corp.chartercom.com] fatal exception while booting Elasticsearch
java.lang.ExceptionInInitializerError: null
        at org.elasticsearch.bootstrap.JNANatives.definitelyRunningAsRoot(JNANatives.java:162) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Natives.definitelyRunningAsRoot(Natives.java:57) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initializeNatives(Elasticsearch.java:259) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.initPhase2(Elasticsearch.java:166) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:66) ~[elasticsearch-8.4.1.jar:?]
Caused by: java.lang.UnsupportedOperationException: Failed to allocate closure
        at com.sun.jna.Native.registerMethod(Native Method) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1906) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1775) ~[jna-5.10.0.jar:?]
        at com.sun.jna.Native.register(Native.java:1493) ~[jna-5.10.0.jar:?]
        at org.elasticsearch.bootstrap.JNACLibrary.<clinit>(JNACLibrary.java:38) ~[elasticsearch-8.4.1.jar:?]
        ... 5 more

This suggests that something has asked Elasticsearch to shutdown.

I would look in your OS logs to see if there is anything related.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.