Elastic search server startup fails with "java.lang.ArrayIndexOutOfBoundException" in 6.2

Elastic Stack Elasticsearch
1

I have a 5-node cluster running in production already in GCP. Since the load has increased, I tried to add few more nodes to that cluster. To create new nodes, I used "Create similar" option provided by GCP. I updated all configurations like node name and deleted the /var/lib/elasticsearch/nodes folder and tried to start the ES server on the new node. But it always fails with following error :

[2019-06-27T00:16:01,471][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-10] fatal error in thread [main], exiting
java.lang.ExceptionInInitializerError: null
    at java.lang.Class.forName0(Native Method) ~[?:1.8.0_212]
    at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_212]
    at org.elasticsearch.painless.Definition.addStruct(Definition.java:753) ~[?:?]
    at org.elasticsearch.painless.Definition.<init>(Definition.java:566) ~[?:?]
    at org.elasticsearch.painless.PainlessScriptEngine.<init>(PainlessScriptEngine.java:106) ~[?:?]
    at org.elasticsearch.painless.PainlessPlugin.getScriptEngine(PainlessPlugin.java:59) ~[?:?]
    at org.elasticsearch.script.ScriptModule.<init>(ScriptModule.java:69) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.node.Node.<init>(Node.java:327) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.node.Node.<init>(Node.java:246) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:213) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.2.2.jar:6.2.2]
    at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-6.2.2.jar:6.2.2]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) ~[elasticsearch-6.2.2.jar:6.2.2]
Caused by: java.lang.ArrayIndexOutOfBoundsException: 4
		at java.time.chrono.JapaneseEra.<clinit>(JapaneseEra.java:179) ~[?:1.8.0_212]
		... 19 more

This node uses OpenJDK 1.8. I enabled trace log for root logger, but couldn't identify what's wrong with the new node.

Please help in identifying what's the root cause of this problem.

2

Looks like a duplicate of this issue:

I think this indicates that your JDK is not properly installed, and is using code with a mix of versions.

3

Thank you for your quick response David!

I followed the suggestion mentioned in the linked issue. I tried to uninstall OpenJDK using the command "sudo apt-get purge --auto-remove openjdk*" and saw an error message "E: dpkg was interrupted, you must manually run 'sudo dpkg --configure -a' to correct the problem."

So I ran the command "sudo dpkg --configure -a" to correct this problem. After I ran this command, OpenJDK was still installed and later when I started ES server, it started without any errors!! I don't know what made the difference.

I wonder why the existing servers work fine and the new node which is cloned from the same base image does not work? The JDK installation is same in both cases, but the new one refused to start until I ran these commands.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.