ES 5.3 : Java client hangs on new PreBuiltTransportClient

Client hangs on calling the constructor

    public static void main(String[] args) throws IOException,Exception {

    Settings setting = Settings.builder()
            .put("cluster.name", "elasticsearch")
            .put("client.transport.sniff", false)
            .build();

    PreBuiltTransportClient client  = null;
    try {
        // Hangs here
        client = new PreBuiltTransportClient(setting);
    }
    catch (Exception e){

        System.out.println("s");
    }


}

My gradle deps

  compile (
        'org.elasticsearch:elasticsearch:5.3.0',
        'org.elasticsearch.client:transport:5.3.0',
        'org.apache.logging.log4j:log4j-api:2.8',
        'org.apache.logging.log4j:log4j-core:2.8',

)

Jstack output

Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.66-b17 mixed mode):

"Attach Listener" #15 daemon prio=9 os_prio=0 tid=0x00007fbb50001000 nid=0x2414 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"elasticsearch[client][[timer]]" #14 daemon prio=5 os_prio=0 tid=0x00007fbb9497b000 nid=0x23a5 waiting on condition [0x00007fbb7b3cd000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.elasticsearch.threadpool.ThreadPool$CachedTimeThread.run(ThreadPool.java:536)

"Service Thread" #11 daemon prio=9 os_prio=0 tid=0x00007fbb94469800 nid=0x239d runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C1 CompilerThread2" #10 daemon prio=9 os_prio=0 tid=0x00007fbb94466000 nid=0x239c waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" #9 daemon prio=9 os_prio=0 tid=0x00007fbb94462000 nid=0x239b waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #8 daemon prio=9 os_prio=0 tid=0x00007fbb94460800 nid=0x239a waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"JDWP Command Reader" #7 daemon prio=10 os_prio=0 tid=0x00007fbb44001000 nid=0x2396 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"JDWP Event Helper Thread" #6 daemon prio=10 os_prio=0 tid=0x00007fbb94196800 nid=0x2395 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"JDWP Transport Listener: dt_socket" #5 daemon prio=10 os_prio=0 tid=0x00007fbb94193000 nid=0x2394 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007fbb94184800 nid=0x2392 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007fbb9414d800 nid=0x2391 in Object.wait() [0x00007fbb80f17000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x000000075dd80db0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x000000075dd80db0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007fbb9414b800 nid=0x2390 in Object.wait() [0x00007fbb81018000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x000000075dd80df0> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
- locked <0x000000075dd80df0> (a java.lang.ref.Reference$Lock)

"main" #1 prio=5 os_prio=0 tid=0x00007fbb94011000 nid=0x238a runnable [0x00007fbb9b8fe000]
java.lang.Thread.State: RUNNABLE
at sun.nio.fs.UnixNativeDispatcher.stat0(Native Method)
at sun.nio.fs.UnixNativeDispatcher.stat(UnixNativeDispatcher.java:286)
at sun.nio.fs.UnixFileAttributes.get(UnixFileAttributes.java:70)
at sun.nio.fs.UnixFileStore.devFor(UnixFileStore.java:55)
at sun.nio.fs.UnixFileStore.(UnixFileStore.java:70)
at sun.nio.fs.LinuxFileStore.(LinuxFileStore.java:48)
at sun.nio.fs.LinuxFileSystem.getFileStore(LinuxFileSystem.java:112)
at sun.nio.fs.UnixFileSystem$FileStoreIterator.readNext(UnixFileSystem.java:213)
at sun.nio.fs.UnixFileSystem$FileStoreIterator.hasNext(UnixFileSystem.java:224)
- locked <0x0000000759da8b28> (a sun.nio.fs.UnixFileSystem$FileStoreIterator)
at org.apache.lucene.util.IOUtils.getFileStore(IOUtils.java:543)
at org.apache.lucene.util.IOUtils.spinsLinux(IOUtils.java:487)
at org.apache.lucene.util.IOUtils.spins(IOUtils.java:476)
at org.elasticsearch.env.ESFileStore.(ESFileStore.java:60)
at org.elasticsearch.env.Environment.(Environment.java:105)
at org.elasticsearch.common.settings.ClusterSettings.(ClusterSettings.java:167)
at org.elasticsearch.common.settings.SettingsModule.(SettingsModule.java:66)
at org.elasticsearch.client.transport.TransportClient.buildTemplate(TransportClient.java:140)
at org.elasticsearch.client.transport.TransportClient.(TransportClient.java:268)
at org.elasticsearch.transport.client.PreBuiltTransportClient.(PreBuiltTransportClient.java:125)
at org.elasticsearch.transport.client.PreBuiltTransportClient.(PreBuiltTransportClient.java:111)
at org.elasticsearch.transport.client.PreBuiltTransportClient.(PreBuiltTransportClient.java:101)
at Entry.main(Entry.java:18)

"VM Thread" os_prio=0 tid=0x00007fbb94146000 nid=0x238f runnable

"GC task thread#0 (ParallelGC)" os_prio=0 tid=0x00007fbb94026800 nid=0x238b runnable

"GC task thread#1 (ParallelGC)" os_prio=0 tid=0x00007fbb94028000 nid=0x238c runnable

"GC task thread#2 (ParallelGC)" os_prio=0 tid=0x00007fbb9402a000 nid=0x238d runnable

"GC task thread#3 (ParallelGC)" os_prio=0 tid=0x00007fbb9402b800 nid=0x238e runnable

"VM Periodic Task Thread" os_prio=0 tid=0x00007fbb9446f000 nid=0x239e waiting on condition

JNI global references: 7761

Just to make sure. It hangs forever or for 30s then is failing ?

Is elasticsearch running?

It hangs "forever" which is to say I gave up on it after 5-10 mins.

I discovered the issue.

I saw a similar issue discussed on a github ticket but initially didn't pay much attention to since it seemed relevant to the server and not the TransportClient.

The issue was I have a mount point whose filesystem (NFS) was not mounted. It appears the client attempts to enumerate mount points. Because the filesystem wasn't mounted the client was hanging indefinitely.

A line or two of documentation could mention this.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.