Java Transport Client 5.4.1 NullPointerException

aszac · June 7, 2017, 1:17pm

Hello,

I am trying to upgrade my service from Elasticsearch 1.4 to Elasticsearch 5.4.1. It run without issues on Centos 6.8, but after the upgrade I am getting the following error:

Exception in thread "main" java.lang.ExceptionInInitializerError
at org.elasticsearch.common.logging.DeprecationLogger.(DeprecationLogger.java:138)
at org.elasticsearch.common.xcontent.support.AbstractXContentParser.(AbstractXContentParser.java:57)
at org.elasticsearch.common.xcontent.json.JsonXContentParser.(JsonXContentParser.java:44)
at org.elasticsearch.common.xcontent.json.JsonXContent.createParser(JsonXContent.java:103)
at org.elasticsearch.common.settings.Setting.parseableStringToList(Setting.java:848)
at org.elasticsearch.common.settings.Setting.lambda$listSetting$27(Setting.java:802)
at org.elasticsearch.common.settings.Setting.listSetting(Setting.java:807)
at org.elasticsearch.common.settings.Setting.listSetting(Setting.java:802)
at org.elasticsearch.common.network.NetworkService.(NetworkService.java:50)
at org.elasticsearch.client.transport.TransportClient.newPluginService(TransportClient.java:98)
at org.elasticsearch.client.transport.TransportClient.buildTemplate(TransportClient.java:126)
at org.elasticsearch.client.transport.TransportClient.(TransportClient.java:268)
at org.elasticsearch.transport.client.PreBuiltTransportClient.(PreBuiltTransportClient.java:127)
at org.elasticsearch.transport.client.PreBuiltTransportClient.(PreBuiltTransportClient.java:113)
at org.elasticsearch.transport.client.PreBuiltTransportClient.(PreBuiltTransportClient.java:103)
...
Caused by: java.lang.NullPointerException
at org.elasticsearch.Build.(Build.java:51)
... 24 more

What could be the cause of this problem?

Thanks,

Agnieszka

dadoonet · June 7, 2017, 1:46pm

Never saw that. It sounds like the manifest file is not there.
My guess is that you still have an old elasticsearch jar somewhere in your class path.

aszac · June 7, 2017, 2:42pm

My class path has elasticsearch-5.4.1.jar and transport-5.4.1.jar and I do have a manifest file present.

aszac · June 7, 2017, 3:20pm

There has been a thread about this issue already, but apparently it has been resolved: ES 5.3 and Java-API 5.3.0 > java.lang.NullPointerException? in Elasticsearch 5.3. However, I am still seeing that issue the the latest version (5.4.1 and 5.4.0).

dadoonet · June 7, 2017, 4:34pm

@jasontedor WDYT?

jcalonso · June 8, 2017, 3:17pm

To complement the information on this issue, we found that:

github.com

elastic/elasticsearch/blob/v5.4.1/core/src/main/java/org/elasticsearch/Build.java#L49




static {
    final String shortHash;
    final String date;
    final boolean isSnapshot;


    final String esPrefix = "elasticsearch-" + Version.CURRENT;
    final URL url = getElasticsearchCodebase();
    final String urlStr = url.toString();
    if (urlStr.startsWith("file:/") && (urlStr.endsWith(esPrefix + ".jar") || urlStr.endsWith(esPrefix + "-SNAPSHOT.jar"))) {
        try (JarInputStream jar = new JarInputStream(url.openStream())) {
            Manifest manifest = jar.getManifest();
            shortHash = manifest.getMainAttributes().getValue("Change");
            date = manifest.getMainAttributes().getValue("Build-Date");
            isSnapshot = "true".equals(manifest.getMainAttributes().getValue("X-Compile-Elasticsearch-Snapshot"));
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    } else {
        // not running from the official elasticsearch jar file (unit tests, IDE, uber client jar, shadiness)
        shortHash = "Unknown";

Returns a JarInputStream with man = null (and most of the other properties are null) possibly because is not being able to read the MANIFEST file.

To verify that there was a Manifest file we manually uncompress the elasticsearch-5.4.1.jar file and the file seems to be correct.

Any ideas what can cause this behaviour?

If it helps:
OS: Centos 6.8
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (build 1.8.0_121-b13)
OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)

rjernst · June 8, 2017, 7:18pm

How are you starting elasticsearch?

aszac · June 8, 2017, 7:37pm

We are starting the elasticsearch with the following config:

cluster.name: "elasticsearch"
node.master: true
node.data: true
path.data: /var/log/elasticsearch/
path.logs: /var/log/elasticsearch/
node.name: "test"

The command is:
/usr/share/elasticsearch/bin/elasticsearch -Epath.conf=/etc/elasticsearch/elasticsearch.yml

jasontedor · June 11, 2017, 12:00pm

How are you starting the transport client, what is it hosted inside of?

jcalonso · June 12, 2017, 6:04pm

Hi jasontedor, here is a snippet:

 val s = Settings.builder()
    .put("cluster.name", clusterName)
    .build()

  val client = new PreBuiltTransportClient(s)
  nodes.asScala.foreach { node =>
    client.addTransportAddress(new InetSocketTransportAddress(node))
  }

The exception occurs on the instantiation of the PreBuiltTransportClient

We found that our elasticsearch-5.4.1.jar has a different sha1 hash from the one published on maven central.

We are investigating this issue, but I believe this is the reason why it cannot read the manifest file.

jasontedor · June 13, 2017, 3:51am

That would indeed be a problem.

billnbell · June 19, 2017, 4:59pm

Has this been fixed?

jasontedor · June 19, 2017, 7:01pm

@billnbell Is what fixed? The issue reported here for this user was caused by them not using an official distribution.

jcalonso · June 20, 2017, 7:55pm

@jasontedor we are using the official distribution of Elasticsearch.
After more investigation we found that the issue was in our CI system at build time when it create rpms for our Scala services.

Apparently there is some jar repackaging happening on Centos that changes the sha1 sums of the jar files and in the case of elasticsearch-5.4.1.jar when you try to instantiate a PreBuiltTransportClient it does some validation using the MANIFEST file and this was failing because of the different sha1sum.

To fix this we had to explicit remove the repackaging in the spec files generated by sbt and maven.

For sbt (Build.sbt):

import com.typesafe.sbt.packager.rpm.RpmPlugin.Names
val script =
  """%define __os_install_post \
%{_rpmconfigdir}/brp-compress \
%{!?__debug_package:%{_rpmconfigdir}/brp-strip %{__strip}} \
%{_rpmconfigdir}/brp-strip-static-archive %{__strip} \
%{_rpmconfigdir}/brp-strip-comment-note %{__strip} %{__objdump} \
%{nil}
"""
...


maintainerScripts in Rpm += (Names.Pre -> ((maintainerScripts in Rpm).value.getOrElse(Names.Pre, Nil) :+ script))

For maven (inside the rpm-maven-plugin configuration node) :

                            <defineStatements>
                                <defineStatement>_unpackaged_files_terminate_build 0</defineStatement>
                                <defineStatement>__os_install_post \
%{_rpmconfigdir}/brp-compress \
%{!?__debug_package:%{_rpmconfigdir}/brp-strip %{__strip}} \
%{_rpmconfigdir}/brp-strip-static-archive %{__strip} \
%{_rpmconfigdir}/brp-strip-comment-note %{__strip} %{__objdump} \
%{nil}
                                </defineStatement>
</defineStatements>

jasontedor · June 21, 2017, 10:24am

I'm glad you've found the issue. I want to be clear about something for others that are following this thread. A repackaged distribution is not an official distribution, and we do not support repackaged distributions.

jcalonso · June 21, 2017, 10:39am

Thanks the help and clarification @jasontedor!

system · July 19, 2017, 10:39am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
6.0.0: NullPointerException at org.elasticsearch.Build.<clinit>(Build.java:49) Elasticsearch	7	1459	January 4, 2018
NullPointerException when build a transportclient with version 2.3.2 Elasticsearch	2	785	March 27, 2017
ES 5.3 and Java-API 5.3.0 > java.lang.NullPointerException? Elasticsearch	17	4144	June 10, 2017
Building Elasticsearch for Clear Linux, NullPointerException Elasticsearch	7	1304	July 12, 2017
NullPointerException in scroll Elasticsearch	10	2239	October 5, 2017

Java Transport Client 5.4.1 NullPointerException

Related topics