Embedded es test server hangs on startup

Elastic Stack Elasticsearch
1

I'm trying run elasticsearch as part of my jruby tests. Here's some of the
code I use to do that:

        Settings settings = ImmutableSettings.settingsBuilder()

                .put("name", nodeName)

                .put("cluster.name", "linko-dev-cluster")

                .put("index.gateway.type", "none")

                .put("gateway.type", "none")

                .put("discovery.zen.ping.multicast.enabled", "false")

                .put("path.data", indexDir)

                .put("path.logs", logDir)

                .put("foreground", "true")

                .put("http.port", esPort)

                .build();



        NodeBuilder nodeBuilder = NodeBuilder.nodeBuilder()

                .settings(settings)

                .loadConfigSettings(false);

        node = nodeBuilder

                .build();

        // register a shutdown hook

        Runtime.getRuntime().addShutdownHook(new Thread() {

            @Override

            public void run() {

                node.close();

            }

        });

        node.start();

From my IDE and maven this works fine but when I call this from within
jruby in my rspec, elasticsearch hangs when I try to start it. A kill -QUIT
shows that it hangs on some initialization code related to tribe
functionality:

"main" #1 prio=5 os_prio=31 tid=0x00007fefe9801000 nid=0x1903 waiting on
condition [0x0000000108eb8000]

java.lang.Thread.State: WAITING (parking)

at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000795b61a58> (a
    java.util.concurrent.CountDownLatch$Sync)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)

at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)

at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)

at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)

at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)

at org.elasticsearch.tribe.TribeService.doStart(TribeService.java:171)

at
org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)

at org.elasticsearch.node.internal.InternalNode.start(InternalNode.java:240)

at io.linko.ng.es.EsTestLauncher.start(EsTestLauncher.java:108)

I'd really appreciate any workarounds. I don't really need the transport
running or tribe. I just need to be able to connect via http. I'm using
jruby 1.7.12, java 1.8, and elasticsearch 1.1.1 pulled in via a maven
dependency.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/51c219a1-e96b-4f5c-81b5-d8728c7471e3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Does it work better if you build the node with local(true) ?

Something like

Node node = nodeBuilder().local(true).settings(settings).build();

For junit testing I also use randomized cluster names for each node.

Otherwise I get a bunch of nodes on a single cluster on the JVM from the
previous runs that do not respond and take long time before timing out. Not
sure what causes this, maybe also the TribeService startup.

To activate tribe node, ES should add a global setting like "tribe.enabled
: false/true"

Jörg

On Thu, May 8, 2014 at 2:59 PM, Jilles van Gurp jillesvangurp@gmail.comwrote:

I'm trying run elasticsearch as part of my jruby tests. Here's some of the
code I use to do that:

        Settings settings = ImmutableSettings.settingsBuilder()

                .put("name", nodeName)

                .put("cluster.name", "linko-dev-cluster")

                .put("index.gateway.type", "none")

                .put("gateway.type", "none")

                .put("discovery.zen.ping.multicast.enabled", "false")

                .put("path.data", indexDir)

                .put("path.logs", logDir)

                .put("foreground", "true")

                .put("http.port", esPort)

                .build();



        NodeBuilder nodeBuilder = NodeBuilder.nodeBuilder()

                .settings(settings)

                .loadConfigSettings(false);

        node = nodeBuilder

                .build();

        // register a shutdown hook

        Runtime.getRuntime().addShutdownHook(new Thread() {

            @Override

            public void run() {

                node.close();

            }

        });

        node.start();

From my IDE and maven this works fine but when I call this from within
jruby in my rspec, elasticsearch hangs when I try to start it. A kill -QUIT
shows that it hangs on some initialization code related to tribe
functionality:

"main" #1 prio=5 os_prio=31 tid=0x00007fefe9801000 nid=0x1903 waiting on
condition [0x0000000108eb8000]

java.lang.Thread.State: WAITING (parking)

at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000795b61a58> (a
    java.util.concurrent.CountDownLatch$Sync)

at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)

at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)

at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)

at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)

at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)

at org.elasticsearch.tribe.TribeService.doStart(TribeService.java:171)

at
org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)

at
org.elasticsearch.node.internal.InternalNode.start(InternalNode.java:240)

at io.linko.ng.es.EsTestLauncher.start(EsTestLauncher.java:108)

I'd really appreciate any workarounds. I don't really need the transport
running or tribe. I just need to be able to connect via http. I'm using
jruby 1.7.12, java 1.8, and elasticsearch 1.1.1 pulled in via a maven
dependency.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/51c219a1-e96b-4f5c-81b5-d8728c7471e3%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/51c219a1-e96b-4f5c-81b5-d8728c7471e3%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH8UP9xUz0w3Z716UXo9sAKhaPWkTZ94mYtGh%2BmUqFYbQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Thanks, I've added local=true and randomizing the name seems also a good
idea.

This seems to move the problem to the DiscoveryService.

Here's the full thread dump:

Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.0-b70 mixed mode):

"elasticsearch[devnode][clusterService#updateTask][T#1]" #26 daemon prio=5
os_prio=31 tid=0x00007ff3d3523000 nid=0x7503 waiting for monitor entry
[0x00000001204f4000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:335)
at
org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:134)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:744)

"elasticsearch[devnode][[ttl_expire]]" #24 daemon prio=5 os_prio=31
tid=0x00007ff3d5722000 nid=0x7303 waiting for monitor entry
[0x00000001200d6000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.elasticsearch.indices.ttl.IndicesTTLService$Notifier.await(IndicesTTLService.java:324)
at
org.elasticsearch.indices.ttl.IndicesTTLService$PurgerThread.run(IndicesTTLService.java:146)

"elasticsearch[devnode][scheduler][T#1]" #23 daemon prio=5 os_prio=31
tid=0x00007ff3d3dce000 nid=0x7103 runnable [0x000000011fe57000]
java.lang.Thread.State: RUNNABLE
at sun.management.MemoryPoolImpl.getUsage0(Native Method)
at sun.management.MemoryPoolImpl.getUsage(MemoryPoolImpl.java:96)
at org.elasticsearch.monitor.jvm.JvmStats.jvmStats(JvmStats.java:151)
at
org.elasticsearch.monitor.jvm.JvmMonitorService$JvmMonitor.monitorLongGc(JvmMonitorService.java:167)

  • locked <0x00000007969d3f10> (a
    org.elasticsearch.monitor.jvm.JvmMonitorService$JvmMonitor)
    at
    org.elasticsearch.monitor.jvm.JvmMonitorService$JvmMonitor.run(JvmMonitorService.java:159)
    at
    org.elasticsearch.threadpool.ThreadPool$LoggingRunnable.run(ThreadPool.java:437)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at
    java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at
    java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:744)

"elasticsearch[devnode][[timer]]" #22 daemon prio=5 os_prio=31
tid=0x00007ff3d3954800 nid=0x6f07 waiting on condition [0x000000011fc54000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.elasticsearch.threadpool.ThreadPool$EstimatedTimeThread.run(ThreadPool.java:514)

"AsyncAppender-Worker-Thread-8" #21 daemon prio=5 os_prio=31
tid=0x00007ff3d42af800 nid=0x6603 waiting for monitor entry
[0x000000011ed5c000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
ch.qos.logback.core.pattern.FormattingConverter.write(FormattingConverter.java:66)
at
ch.qos.logback.core.pattern.CompositeConverter.convert(CompositeConverter.java:24)
at
ch.qos.logback.core.pattern.FormattingConverter.write(FormattingConverter.java:37)
at
ch.qos.logback.core.pattern.PatternLayoutBase.writeLoopOnConverters(PatternLayoutBase.java:119)
at ch.qos.logback.classic.PatternLayout.doLayout(PatternLayout.java:149)
at ch.qos.logback.classic.PatternLayout.doLayout(PatternLayout.java:39)
at
ch.qos.logback.core.encoder.LayoutWrappingEncoder.doEncode(LayoutWrappingEncoder.java:134)
at
ch.qos.logback.core.OutputStreamAppender.writeOut(OutputStreamAppender.java:188)
at ch.qos.logback.core.FileAppender.writeOut(FileAppender.java:206)
at
ch.qos.logback.core.OutputStreamAppender.subAppend(OutputStreamAppender.java:212)

  • locked <0x0000000740becbb0> (a ch.qos.logback.core.spi.LogbackLock)
    at
    ch.qos.logback.core.rolling.RollingFileAppender.subAppend(RollingFileAppender.java:175)
    at
    ch.qos.logback.core.OutputStreamAppender.append(OutputStreamAppender.java:103)
    at
    ch.qos.logback.core.UnsynchronizedAppenderBase.doAppend(UnsynchronizedAppenderBase.java:88)
    at
    ch.qos.logback.core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:48)
    at
    ch.qos.logback.core.AsyncAppenderBase$Worker.run(AsyncAppenderBase.java:226)

"AsyncAppender-Worker-Thread-7" #20 daemon prio=5 os_prio=31
tid=0x00007ff3d51fe800 nid=0x6403 waiting on condition [0x000000011e6c2000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000740c23378> (a
    java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at
    java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:403)
    at
    ch.qos.logback.core.AsyncAppenderBase$Worker.run(AsyncAppenderBase.java:225)

"AsyncAppender-Worker-Thread-6" #19 daemon prio=5 os_prio=31
tid=0x00007ff3d51fe000 nid=0x6203 waiting for monitor entry
[0x000000011e4be000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.lang.ClassLoader.loadClass(ClassLoader.java:404)

  • waiting to lock <0x0000000740095588> (a org.jruby.util.JRubyClassLoader)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at
    ch.qos.logback.core.util.OptionHelper.instantiateByClassNameAndParameter(OptionHelper.java:60)
    at
    ch.qos.logback.core.util.OptionHelper.instantiateByClassName(OptionHelper.java:48)
    at
    ch.qos.logback.core.util.OptionHelper.instantiateByClassName(OptionHelper.java:35)
    at
    ch.qos.logback.core.joran.action.AppenderAction.begin(AppenderAction.java:54)
    at
    ch.qos.logback.core.joran.spi.Interpreter.callBeginAction(Interpreter.java:275)
    at
    ch.qos.logback.core.joran.spi.Interpreter.startElement(Interpreter.java:147)
    at
    ch.qos.logback.core.joran.spi.Interpreter.startElement(Interpreter.java:129)
    at ch.qos.logback.core.joran.spi.EventPlayer.play(EventPlayer.java:50)
    at
    ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:149)
  • locked <0x0000000796d4f0e0> (a ch.qos.logback.core.spi.LogbackLock)
    at
    ch.qos.logback.core.sift.SiftingJoranConfiguratorBase.doConfigure(SiftingJoranConfiguratorBase.java:85)
    at
    ch.qos.logback.core.sift.AbstractAppenderFactoryUsingJoran.buildAppender(AbstractAppenderFactoryUsingJoran.java:51)
    at
    ch.qos.logback.core.sift.AppenderTracker.buildComponent(AppenderTracker.java:57)
    at
    ch.qos.logback.core.sift.AppenderTracker.buildComponent(AppenderTracker.java:32)
    at
    ch.qos.logback.core.spi.AbstractComponentTracker.getOrCreate(AbstractComponentTracker.java:124)
  • locked <0x0000000740c23b40> (a ch.qos.logback.core.sift.AppenderTracker)
    at
    ch.qos.logback.core.sift.SiftingAppenderBase.append(SiftingAppenderBase.java:105)
    at ch.qos.logback.core.AppenderBase.doAppend(AppenderBase.java:85)
  • locked <0x0000000740c23a68> (a
    ch.qos.logback.classic.sift.SiftingAppender)
    at
    ch.qos.logback.core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:48)
    at
    ch.qos.logback.core.AsyncAppenderBase$Worker.run(AsyncAppenderBase.java:226)

"AsyncAppender-Worker-Thread-5" #18 daemon prio=5 os_prio=31
tid=0x00007ff3d54ba800 nid=0x6003 waiting for monitor entry
[0x000000011e2bc000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
ch.qos.logback.core.pattern.FormattingConverter.write(FormattingConverter.java:66)
at
ch.qos.logback.core.pattern.CompositeConverter.convert(CompositeConverter.java:24)
at
ch.qos.logback.core.pattern.FormattingConverter.write(FormattingConverter.java:37)
at
ch.qos.logback.core.pattern.PatternLayoutBase.writeLoopOnConverters(PatternLayoutBase.java:119)
at ch.qos.logback.classic.PatternLayout.doLayout(PatternLayout.java:149)
at ch.qos.logback.classic.PatternLayout.doLayout(PatternLayout.java:39)
at
ch.qos.logback.core.encoder.LayoutWrappingEncoder.doEncode(LayoutWrappingEncoder.java:134)
at
ch.qos.logback.core.OutputStreamAppender.writeOut(OutputStreamAppender.java:188)
at
ch.qos.logback.core.OutputStreamAppender.subAppend(OutputStreamAppender.java:212)

  • locked <0x0000000796d01e60> (a ch.qos.logback.core.spi.LogbackLock)
    at
    ch.qos.logback.core.OutputStreamAppender.append(OutputStreamAppender.java:103)
    at
    ch.qos.logback.core.UnsynchronizedAppenderBase.doAppend(UnsynchronizedAppenderBase.java:88)
    at
    ch.qos.logback.core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:48)
    at
    ch.qos.logback.core.AsyncAppenderBase$Worker.run(AsyncAppenderBase.java:226)

"NonBlockingInputStreamThread" #17 daemon prio=5 os_prio=31
tid=0x00007ff3d581c800 nid=0x440b in Object.wait() [0x000000011c610000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)

  • waiting on <0x0000000796e2b480> (a jline.internal.NonBlockingInputStream)
    at
    jline.internal.NonBlockingInputStream.run(NonBlockingInputStream.java:278)
  • locked <0x0000000796e2b480> (a jline.internal.NonBlockingInputStream)
    at java.lang.Thread.run(Thread.java:744)

"process reaper" #15 daemon prio=10 os_prio=31 tid=0x00007ff3d6c08800
nid=0x3b07 waiting on condition [0x000000011aa14000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000740506940> (a
    java.util.concurrent.SynchronousQueue$TransferStack)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
    at
    java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
    at
    java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
    at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
    at
    java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
    at
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:744)

"Ruby-0-JIT-2" #14 daemon prio=1 os_prio=31 tid=0x00007ff3d28e5000
nid=0x5d03 waiting on condition [0x000000011d2da000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000740095c68> (a
    java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at
    java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at
    java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at
    java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
    at
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:744)

"Ruby-0-JIT-1" #11 daemon prio=1 os_prio=31 tid=0x00007ff3d2915000
nid=0x5b03 waiting on condition [0x000000011ccda000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000740095c68> (a
    java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at
    java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at
    java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at
    java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
    at
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:744)

"Service Thread" #9 daemon prio=9 os_prio=31 tid=0x00007ff3d302a000
nid=0x5703 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C1 CompilerThread3" #8 daemon prio=9 os_prio=31 tid=0x00007ff3d2842800
nid=0x5503 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C2 CompilerThread2" #7 daemon prio=9 os_prio=31 tid=0x00007ff3d2842000
nid=0x5303 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" #6 daemon prio=9 os_prio=31 tid=0x00007ff3d2841000
nid=0x5103 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #5 daemon prio=9 os_prio=31 tid=0x00007ff3d5807000
nid=0x4f03 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" #4 daemon prio=9 os_prio=31 tid=0x00007ff3d5801800
nid=0x4d03 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Finalizer" #3 daemon prio=8 os_prio=31 tid=0x00007ff3d4007000 nid=0x3903
in Object.wait() [0x000000011a9be000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142)

  • locked <0x00000007401b7d18> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158)
    at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

"Reference Handler" #2 daemon prio=10 os_prio=31 tid=0x00007ff3d5010000
nid=0x3703 in Object.wait() [0x000000011a7bb000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)

  • locked <0x00000007401325e8> (a java.lang.ref.Reference$Lock)

"main" #1 prio=5 os_prio=31 tid=0x00007ff3d280a000 nid=0x1903 waiting on
condition [0x00000001036b8000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x00000007969d5138> (a
    java.util.concurrent.CountDownLatch$Sync)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
    at
    java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
    at
    java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
    at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
    at
    org.elasticsearch.discovery.DiscoveryService.doStart(DiscoveryService.java:68)
    at
    org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)
    at org.elasticsearch.node.internal.InternalNode.start(InternalNode.java:230)
    at io.linko.ng.es.EsTestLauncher.start(EsTestLauncher.java:115)
    at
    io.linko.ng.es.EsTestLauncher.ensureElasticSearchIsUp(EsTestLauncher.java:49)
  • locked <0x0000000740095588> (a org.jruby.util.JRubyClassLoader)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at
    org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:440)
    at org.jruby.javasupport.JavaMethod.invokeStaticDirect(JavaMethod.java:362)
    at
    org.jruby.java.invokers.StaticMethodInvoker.call(StaticMethodInvoker.java:50)
    at
    org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)
    at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60)
    at org.jruby.ast.ClassVarAsgnNode.interpret(ClassVarAsgnNode.java:92)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.RootNode.interpret(RootNode.java:129)
    at org.jruby.evaluator.ASTInterpreter.INTERPRET_EVAL(ASTInterpreter.java:95)
    at
    org.jruby.evaluator.ASTInterpreter.evalWithBinding(ASTInterpreter.java:184)
    at org.jruby.RubyKernel.evalCommon(RubyKernel.java:1138)
    at org.jruby.RubyKernel.eval19(RubyKernel.java:1101)
    at
    org.jruby.RubyKernel$INVOKER$s$0$3$eval19.call(RubyKernel$INVOKER$s$0$3$eval19.gen)
    at
    org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:180)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:70)
    at org.jruby.ast.FCallManyArgsNode.interpret(FCallManyArgsNode.java:60)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
    at
    org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:112)
    at
    org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:126)
    at
    org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:173)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:70)
    at org.jruby.ast.CallManyArgsNode.interpret(CallManyArgsNode.java:59)
    at org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
    at
    org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:225)
    at
    org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:214)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:202)
    at org.jruby.ast.CallTwoArgNode.interpret(CallTwoArgNode.java:59)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at org.jruby.ast.RescueNode.executeBody(RescueNode.java:221)
    at org.jruby.ast.RescueNode.interpret(RescueNode.java:116)
    at org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
    at
    org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)
    at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:157)
    at
    org.jruby.runtime.Interpreted19Block.yieldSpecific(Interpreted19Block.java:130)
    at org.jruby.runtime.Block.yieldSpecific(Block.java:111)
    at org.jruby.ast.ZYieldNode.interpret(ZYieldNode.java:25)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.EnsureNode.interpret(EnsureNode.java:96)
    at org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
    at
    org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:204)
    at
    org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:206)
    at
    org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:177)
    at
    org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:188)
    at
    org.jruby.ast.FCallOneArgBlockNode.interpret(FCallOneArgBlockNode.java:34)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
    at
    org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)
    at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:194)
    at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:177)
    at
    org.jruby.runtime.Interpreted19Block.yieldSpecific(Interpreted19Block.java:140)
    at org.jruby.runtime.Block.yieldSpecific(Block.java:129)
    at org.jruby.ast.YieldTwoNode.interpret(YieldTwoNode.java:31)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at org.jruby.ast.IfNode.interpret(IfNode.java:116)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at org.jruby.ast.RescueNode.executeBody(RescueNode.java:221)
    at org.jruby.ast.RescueNode.interpret(RescueNode.java:116)
    at org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
    at
    org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)
    at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:157)
    at
    org.jruby.runtime.Interpreted19Block.yieldSpecific(Interpreted19Block.java:130)
    at org.jruby.runtime.Block.yieldSpecific(Block.java:111)
    at org.jruby.RubyKernel.loop(RubyKernel.java:1501)
    at
    org.jruby.RubyKernel$INVOKER$s$0$0$loop.call(RubyKernel$INVOKER$s$0$0$loop.gen)
    at
    org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:316)
    at
    org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:145)
    at
    org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:154)
    at org.jruby.ast.FCallNoArgBlockNode.interpret(FCallNoArgBlockNode.java:32)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
    at
    org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)
    at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:157)
    at org.jruby.runtime.Block.yield(Block.java:142)
    at org.jruby.RubyContinuation.enter(RubyContinuation.java:107)
    at org.jruby.RubyKernel.rbCatch19Common(RubyKernel.java:1271)
    at org.jruby.RubyKernel.rbCatch19(RubyKernel.java:1264)
    at
    org.jruby.RubyKernel$INVOKER$s$rbCatch19.call(RubyKernel$INVOKER$s$rbCatch19.gen)
    at
    org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:336)
    at
    org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:179)
    at
    org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:188)
    at
    org.jruby.ast.FCallOneArgBlockNode.interpret(FCallOneArgBlockNode.java:34)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
    at
    org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:161)
    at
    org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:190)
    at
    org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:316)
    at
    org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:145)
    at
    org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:154)
    at org.jruby.ast.CallNoArgBlockNode.interpret(CallNoArgBlockNode.java:64)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
    at
    org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:139)
    at
    org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:182)
    at
    org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)
    at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
    at
    org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)
    at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:157)
    at org.jruby.runtime.Block.yield(Block.java:142)
    at org.jruby.RubyContinuation.enter(RubyContinuation.java:107)
    at org.jruby.RubyKernel.rbCatch19Common(RubyKernel.java:1271)
    at org.jruby.RubyKernel.rbCatch19(RubyKernel.java:1264)
    at
    org.jruby.RubyKernel$INVOKER$s$rbCatch19.call(RubyKernel$INVOKER$s$rbCatch19.gen)
    at
    org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:336)
    at
    org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:179)
    at
    org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:188)
    at
    org.jruby.ast.FCallOneArgBlockNode.interpret(FCallOneArgBlockNode.java:34)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.EnsureNode.interpret(EnsureNode.java:96)
    at org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
    at
    org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:182)
    at
    org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:198)
    at
    java.lang.invoke.LambdaForm$DMH/1020923989.invokeVirtual_LLLLLL_L(LambdaForm$DMH)
    at java.lang.invoke.LambdaForm$MH/1458849419.convert(LambdaForm$MH)
    at java.lang.invoke.LambdaForm$MH/600746945.collect(LambdaForm$MH)
    at
    java.lang.invoke.LambdaForm$DMH/2047526627.invokeSpecial_LLLLL_L(LambdaForm$DMH)
    at
    java.lang.invoke.LambdaForm$NamedFunction.invoke_LLLLL_L(LambdaForm.java:1123)
    at
    java.lang.invoke.LambdaForm$DMH/491044090.invokeStatic_LL_L(LambdaForm$DMH)
    at
    java.lang.invoke.LambdaForm$NamedFunction.invokeWithArguments(LambdaForm.java:1147)
    at java.lang.invoke.LambdaForm.interpretName(LambdaForm.java:625)
    at java.lang.invoke.LambdaForm.interpretWithArguments(LambdaForm.java:604)
    at java.lang.invoke.LambdaForm$LFI/1627960023.interpret_L(LambdaForm$LFI)
    at
    java.lang.invoke.LambdaForm$NamedFunction.invoke_LLLLL_L(LambdaForm.java:1123)
    at
    java.lang.invoke.LambdaForm$DMH/491044090.invokeStatic_LL_L(LambdaForm$DMH)
    at
    java.lang.invoke.LambdaForm$NamedFunction.invokeWithArguments(LambdaForm.java:1147)
    at java.lang.invoke.LambdaForm.interpretName(LambdaForm.java:625)
    at java.lang.invoke.LambdaForm.interpretWithArguments(LambdaForm.java:604)
    at java.lang.invoke.LambdaForm$LFI/1627960023.interpret_L(LambdaForm$LFI)
    at
    java.lang.invoke.LambdaForm$NamedFunction.invoke_LLLLL_L(LambdaForm.java:1123)
    at
    java.lang.invoke.LambdaForm$DMH/491044090.invokeStatic_LL_L(LambdaForm$DMH)
    at
    java.lang.invoke.LambdaForm$NamedFunction.invokeWithArguments(LambdaForm.java:1147)
    at java.lang.invoke.LambdaForm.interpretName(LambdaForm.java:625)
    at java.lang.invoke.LambdaForm.interpretWithArguments(LambdaForm.java:604)
    at java.lang.invoke.LambdaForm$LFI/434091818.interpret_L(LambdaForm$LFI)
    at
    java.lang.invoke.LambdaForm$NamedFunction.invoke_LL_L(LambdaForm.java:1108)
    at
    java.lang.invoke.LambdaForm$DMH/491044090.invokeStatic_LL_L(LambdaForm$DMH)
    at
    java.lang.invoke.LambdaForm$NamedFunction.invokeWithArguments(LambdaForm.java:1147)
    at java.lang.invoke.LambdaForm.interpretName(LambdaForm.java:625)
    at java.lang.invoke.LambdaForm.interpretWithArguments(LambdaForm.java:604)
    at java.lang.invoke.LambdaForm$LFI/434091818.interpret_L(LambdaForm$LFI)
    at java.lang.invoke.LambdaForm$MH/1393931310.invokeExact_MT(LambdaForm$MH)
    at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:636)
    at
    org.jruby.runtime.invokedynamic.InvocationLinker.invocationFallback(InvocationLinker.java:157)
    at
    java.lang.invoke.LambdaForm$DMH/895328852.invokeStatic_LLLLL_L(LambdaForm$DMH)
    at
    java.lang.invoke.LambdaForm$DMH/796533847.invokeSpecial_LLLLLL_L(LambdaForm$DMH)
    at java.lang.invoke.LambdaForm$NFI/335471116.invoke_LLLLLL_L(LambdaForm$NFI)
    at
    java.lang.invoke.LambdaForm$DMH/491044090.invokeStatic_LL_L(LambdaForm$DMH)
    at
    java.lang.invoke.LambdaForm$NamedFunction.invokeWithArguments(LambdaForm.java:1147)
    at java.lang.invoke.LambdaForm.interpretName(LambdaForm.java:625)
    at java.lang.invoke.LambdaForm.interpretWithArguments(LambdaForm.java:604)
    at java.lang.invoke.LambdaForm$LFI/672320506.interpret_L(LambdaForm$LFI)
    at java.lang.invoke.LambdaForm$MH/1684106402.linkToCallSite(LambdaForm$MH)
    at
    Users.jilles.$_dot_rvm.rubies.jruby_minus_1_dot_7_dot_12.bin.irb.file(/Users/jilles/.rvm/rubies/jruby-1.7.12/bin/irb:13)
    at
    Users.jilles.$_dot_rvm.rubies.jruby_minus_1_dot_7_dot_12.bin.irb.load(/Users/jilles/.rvm/rubies/jruby-1.7.12/bin/irb)
    at org.jruby.Ruby.runScript(Ruby.java:811)
    at org.jruby.Ruby.runScript(Ruby.java:804)
    at org.jruby.Ruby.runNormally(Ruby.java:673)
    at org.jruby.Ruby.runFromMain(Ruby.java:522)
    at org.jruby.Main.doRunFromMain(Main.java:395)
    at org.jruby.Main.internalRun(Main.java:290)
    at org.jruby.Main.run(Main.java:217)
    at org.jruby.Main.main(Main.java:197)

"VM Thread" os_prio=31 tid=0x00007ff3d500b800 nid=0x3503 runnable

"GC task thread#0 (ParallelGC)" os_prio=31 tid=0x00007ff3d2818000
nid=0x2503 runnable

"GC task thread#1 (ParallelGC)" os_prio=31 tid=0x00007ff3d2818800
nid=0x2703 runnable

"GC task thread#2 (ParallelGC)" os_prio=31 tid=0x00007ff3d2819800
nid=0x2903 runnable

"GC task thread#3 (ParallelGC)" os_prio=31 tid=0x00007ff3d281a000
nid=0x2b03 runnable

"GC task thread#4 (ParallelGC)" os_prio=31 tid=0x00007ff3d281a800
nid=0x2d03 runnable

"GC task thread#5 (ParallelGC)" os_prio=31 tid=0x00007ff3d281b000
nid=0x2f03 runnable

"GC task thread#6 (ParallelGC)" os_prio=31 tid=0x00007ff3d281c000
nid=0x3103 runnable

"GC task thread#7 (ParallelGC)" os_prio=31 tid=0x00007ff3d281c800
nid=0x3303 runnable

"VM Periodic Task Thread" os_prio=31 tid=0x00007ff3d302a800 nid=0x5903
waiting on condition

JNI global references: 4605

Heap
PSYoungGen total 36864K, used 31766K [0x0000000795580000,
0x0000000798a80000, 0x00000007c0000000)
eden space 24064K, 91% used
[0x0000000795580000,0x0000000796b17ac0,0x0000000796d00000)
from space 12800K, 75% used
[0x0000000796d00000,0x000000079766dfe0,0x0000000797980000)
to space 13824K, 0% used
[0x0000000797d00000,0x0000000797d00000,0x0000000798a80000)
ParOldGen total 16896K, used 12528K [0x0000000740000000,
0x0000000741080000, 0x0000000795580000)
object space 16896K, 74% used
[0x0000000740000000,0x0000000740c3c3f8,0x0000000741080000)
Metaspace used 51018K, capacity 55650K, committed 56448K, reserved
1095680K
class space used 8937K, capacity 9852K, committed 10112K, reserved
1048576K

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2f78f672-9017-43c1-85da-0cb02c2d576e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

Found the issue: pilot error.

But thanks for your help.

For the record, I was reusing a launcher class that I use in Java that
works fine from java but fails in ruby. To ensure elasticsearch only
launches once, it uses a synchronized block. Since this is in a static
method, I used a global object to synchronize on: the actual classloader
that loaded the class. I admit this wasn't the brightest of ideas. So,
eventually narrowed it down to that and now lock on a innocent logger
instance. And now it no longer ends up in a deadlock and works fine.

Jilles

On Thursday, May 8, 2014 3:41:23 PM UTC+2, Jilles van Gurp wrote:

Thanks, I've added local=true and randomizing the name seems also a good
idea.

This seems to move the problem to the DiscoveryService.

Here's the full thread dump:

Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.0-b70 mixed mode):

"elasticsearch[devnode][clusterService#updateTask][T#1]" #26 daemon prio=5
os_prio=31 tid=0x00007ff3d3523000 nid=0x7503 waiting for monitor entry
[0x00000001204f4000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:335)
at
org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:134)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:744)

"elasticsearch[devnode][[ttl_expire]]" #24 daemon prio=5 os_prio=31
tid=0x00007ff3d5722000 nid=0x7303 waiting for monitor entry
[0x00000001200d6000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
org.elasticsearch.indices.ttl.IndicesTTLService$Notifier.await(IndicesTTLService.java:324)
at
org.elasticsearch.indices.ttl.IndicesTTLService$PurgerThread.run(IndicesTTLService.java:146)

"elasticsearch[devnode][scheduler][T#1]" #23 daemon prio=5 os_prio=31
tid=0x00007ff3d3dce000 nid=0x7103 runnable [0x000000011fe57000]
java.lang.Thread.State: RUNNABLE
at sun.management.MemoryPoolImpl.getUsage0(Native Method)
at sun.management.MemoryPoolImpl.getUsage(MemoryPoolImpl.java:96)
at org.elasticsearch.monitor.jvm.JvmStats.jvmStats(JvmStats.java:151)
at
org.elasticsearch.monitor.jvm.JvmMonitorService$JvmMonitor.monitorLongGc(JvmMonitorService.java:167)

  • locked <0x00000007969d3f10> (a
    org.elasticsearch.monitor.jvm.JvmMonitorService$JvmMonitor)
    at
    org.elasticsearch.monitor.jvm.JvmMonitorService$JvmMonitor.run(JvmMonitorService.java:159)
    at
    org.elasticsearch.threadpool.ThreadPool$LoggingRunnable.run(ThreadPool.java:437)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at
    java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at
    java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:744)

"elasticsearch[devnode][[timer]]" #22 daemon prio=5 os_prio=31
tid=0x00007ff3d3954800 nid=0x6f07 waiting on condition [0x000000011fc54000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.elasticsearch.threadpool.ThreadPool$EstimatedTimeThread.run(ThreadPool.java:514)

"AsyncAppender-Worker-Thread-8" #21 daemon prio=5 os_prio=31
tid=0x00007ff3d42af800 nid=0x6603 waiting for monitor entry
[0x000000011ed5c000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
ch.qos.logback.core.pattern.FormattingConverter.write(FormattingConverter.java:66)
at
ch.qos.logback.core.pattern.CompositeConverter.convert(CompositeConverter.java:24)
at
ch.qos.logback.core.pattern.FormattingConverter.write(FormattingConverter.java:37)
at
ch.qos.logback.core.pattern.PatternLayoutBase.writeLoopOnConverters(PatternLayoutBase.java:119)
at ch.qos.logback.classic.PatternLayout.doLayout(PatternLayout.java:149)
at ch.qos.logback.classic.PatternLayout.doLayout(PatternLayout.java:39)
at
ch.qos.logback.core.encoder.LayoutWrappingEncoder.doEncode(LayoutWrappingEncoder.java:134)
at
ch.qos.logback.core.OutputStreamAppender.writeOut(OutputStreamAppender.java:188)
at ch.qos.logback.core.FileAppender.writeOut(FileAppender.java:206)
at
ch.qos.logback.core.OutputStreamAppender.subAppend(OutputStreamAppender.java:212)

  • locked <0x0000000740becbb0> (a ch.qos.logback.core.spi.LogbackLock)
    at
    ch.qos.logback.core.rolling.RollingFileAppender.subAppend(RollingFileAppender.java:175)
    at
    ch.qos.logback.core.OutputStreamAppender.append(OutputStreamAppender.java:103)
    at
    ch.qos.logback.core.UnsynchronizedAppenderBase.doAppend(UnsynchronizedAppenderBase.java:88)
    at
    ch.qos.logback.core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:48)
    at
    ch.qos.logback.core.AsyncAppenderBase$Worker.run(AsyncAppenderBase.java:226)

"AsyncAppender-Worker-Thread-7" #20 daemon prio=5 os_prio=31
tid=0x00007ff3d51fe800 nid=0x6403 waiting on condition [0x000000011e6c2000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000740c23378> (a
    java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at
    java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at
    java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:403)
    at
    ch.qos.logback.core.AsyncAppenderBase$Worker.run(AsyncAppenderBase.java:225)

"AsyncAppender-Worker-Thread-6" #19 daemon prio=5 os_prio=31
tid=0x00007ff3d51fe000 nid=0x6203 waiting for monitor entry
[0x000000011e4be000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.lang.ClassLoader.loadClass(ClassLoader.java:404)

  • waiting to lock <0x0000000740095588> (a org.jruby.util.JRubyClassLoader)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at
    ch.qos.logback.core.util.OptionHelper.instantiateByClassNameAndParameter(OptionHelper.java:60)
    at
    ch.qos.logback.core.util.OptionHelper.instantiateByClassName(OptionHelper.java:48)
    at
    ch.qos.logback.core.util.OptionHelper.instantiateByClassName(OptionHelper.java:35)
    at
    ch.qos.logback.core.joran.action.AppenderAction.begin(AppenderAction.java:54)
    at
    ch.qos.logback.core.joran.spi.Interpreter.callBeginAction(Interpreter.java:275)
    at
    ch.qos.logback.core.joran.spi.Interpreter.startElement(Interpreter.java:147)
    at
    ch.qos.logback.core.joran.spi.Interpreter.startElement(Interpreter.java:129)
    at ch.qos.logback.core.joran.spi.EventPlayer.play(EventPlayer.java:50)
    at
    ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:149)
  • locked <0x0000000796d4f0e0> (a ch.qos.logback.core.spi.LogbackLock)
    at
    ch.qos.logback.core.sift.SiftingJoranConfiguratorBase.doConfigure(SiftingJoranConfiguratorBase.java:85)
    at
    ch.qos.logback.core.sift.AbstractAppenderFactoryUsingJoran.buildAppender(AbstractAppenderFactoryUsingJoran.java:51)
    at
    ch.qos.logback.core.sift.AppenderTracker.buildComponent(AppenderTracker.java:57)
    at
    ch.qos.logback.core.sift.AppenderTracker.buildComponent(AppenderTracker.java:32)
    at
    ch.qos.logback.core.spi.AbstractComponentTracker.getOrCreate(AbstractComponentTracker.java:124)
  • locked <0x0000000740c23b40> (a ch.qos.logback.core.sift.AppenderTracker)
    at
    ch.qos.logback.core.sift.SiftingAppenderBase.append(SiftingAppenderBase.java:105)
    at ch.qos.logback.core.AppenderBase.doAppend(AppenderBase.java:85)
  • locked <0x0000000740c23a68> (a
    ch.qos.logback.classic.sift.SiftingAppender)
    at
    ch.qos.logback.core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:48)
    at
    ch.qos.logback.core.AsyncAppenderBase$Worker.run(AsyncAppenderBase.java:226)

"AsyncAppender-Worker-Thread-5" #18 daemon prio=5 os_prio=31
tid=0x00007ff3d54ba800 nid=0x6003 waiting for monitor entry
[0x000000011e2bc000]
java.lang.Thread.State: BLOCKED (on object monitor)
at
ch.qos.logback.core.pattern.FormattingConverter.write(FormattingConverter.java:66)
at
ch.qos.logback.core.pattern.CompositeConverter.convert(CompositeConverter.java:24)
at
ch.qos.logback.core.pattern.FormattingConverter.write(FormattingConverter.java:37)
at
ch.qos.logback.core.pattern.PatternLayoutBase.writeLoopOnConverters(PatternLayoutBase.java:119)
at ch.qos.logback.classic.PatternLayout.doLayout(PatternLayout.java:149)
at ch.qos.logback.classic.PatternLayout.doLayout(PatternLayout.java:39)
at
ch.qos.logback.core.encoder.LayoutWrappingEncoder.doEncode(LayoutWrappingEncoder.java:134)
at
ch.qos.logback.core.OutputStreamAppender.writeOut(OutputStreamAppender.java:188)
at
ch.qos.logback.core.OutputStreamAppender.subAppend(OutputStreamAppender.java:212)

  • locked <0x0000000796d01e60> (a ch.qos.logback.core.spi.LogbackLock)
    at
    ch.qos.logback.core.OutputStreamAppender.append(OutputStreamAppender.java:103)
    at
    ch.qos.logback.core.UnsynchronizedAppenderBase.doAppend(UnsynchronizedAppenderBase.java:88)
    at
    ch.qos.logback.core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:48)
    at
    ch.qos.logback.core.AsyncAppenderBase$Worker.run(AsyncAppenderBase.java:226)

"NonBlockingInputStreamThread" #17 daemon prio=5 os_prio=31
tid=0x00007ff3d581c800 nid=0x440b in Object.wait() [0x000000011c610000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)

  • waiting on <0x0000000796e2b480> (a jline.internal.NonBlockingInputStream)
    at
    jline.internal.NonBlockingInputStream.run(NonBlockingInputStream.java:278)
  • locked <0x0000000796e2b480> (a jline.internal.NonBlockingInputStream)
    at java.lang.Thread.run(Thread.java:744)

"process reaper" #15 daemon prio=10 os_prio=31 tid=0x00007ff3d6c08800
nid=0x3b07 waiting on condition [0x000000011aa14000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000740506940> (a
    java.util.concurrent.SynchronousQueue$TransferStack)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
    at
    java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
    at
    java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
    at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
    at
    java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
    at
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:744)

"Ruby-0-JIT-2" #14 daemon prio=1 os_prio=31 tid=0x00007ff3d28e5000
nid=0x5d03 waiting on condition [0x000000011d2da000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000740095c68> (a
    java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at
    java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at
    java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at
    java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
    at
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:744)

"Ruby-0-JIT-1" #11 daemon prio=1 os_prio=31 tid=0x00007ff3d2915000
nid=0x5b03 waiting on condition [0x000000011ccda000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x0000000740095c68> (a
    java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at
    java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at
    java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at
    java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
    at
    java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
    at
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:744)

"Service Thread" #9 daemon prio=9 os_prio=31 tid=0x00007ff3d302a000
nid=0x5703 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C1 CompilerThread3" #8 daemon prio=9 os_prio=31 tid=0x00007ff3d2842800
nid=0x5503 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C2 CompilerThread2" #7 daemon prio=9 os_prio=31 tid=0x00007ff3d2842000
nid=0x5303 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C2 CompilerThread1" #6 daemon prio=9 os_prio=31 tid=0x00007ff3d2841000
nid=0x5103 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #5 daemon prio=9 os_prio=31 tid=0x00007ff3d5807000
nid=0x4f03 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" #4 daemon prio=9 os_prio=31 tid=0x00007ff3d5801800
nid=0x4d03 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE

"Finalizer" #3 daemon prio=8 os_prio=31 tid=0x00007ff3d4007000 nid=0x3903
in Object.wait() [0x000000011a9be000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142)

  • locked <0x00000007401b7d18> (a java.lang.ref.ReferenceQueue$Lock)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158)
    at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

"Reference Handler" #2 daemon prio=10 os_prio=31 tid=0x00007ff3d5010000
nid=0x3703 in Object.wait() [0x000000011a7bb000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)

  • locked <0x00000007401325e8> (a java.lang.ref.Reference$Lock)

"main" #1 prio=5 os_prio=31 tid=0x00007ff3d280a000 nid=0x1903 waiting on
condition [0x00000001036b8000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)

  • parking to wait for <0x00000007969d5138> (a
    java.util.concurrent.CountDownLatch$Sync)
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
    at
    java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
    at
    java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
    at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
    at
    org.elasticsearch.discovery.DiscoveryService.doStart(DiscoveryService.java:68)
    at
    org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:85)
    at
    org.elasticsearch.node.internal.InternalNode.start(InternalNode.java:230)
    at io.linko.ng.es.EsTestLauncher.start(EsTestLauncher.java:115)
    at
    io.linko.ng.es.EsTestLauncher.ensureElasticSearchIsUp(EsTestLauncher.java:49)
  • locked <0x0000000740095588> (a org.jruby.util.JRubyClassLoader)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at
    org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:440)
    at org.jruby.javasupport.JavaMethod.invokeStaticDirect(JavaMethod.java:362)
    at
    org.jruby.java.invokers.StaticMethodInvoker.call(StaticMethodInvoker.java:50)
    at
    org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)
    at
    org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)
    at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60)
    at org.jruby.ast.ClassVarAsgnNode.interpret(ClassVarAsgnNode.java:92)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.RootNode.interpret(RootNode.java:129)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_EVAL(ASTInterpreter.java:95)
    at
    org.jruby.evaluator.ASTInterpreter.evalWithBinding(ASTInterpreter.java:184)
    at org.jruby.RubyKernel.evalCommon(RubyKernel.java:1138)
    at org.jruby.RubyKernel.eval19(RubyKernel.java:1101)
    at
    org.jruby.RubyKernel$INVOKER$s$0$3$eval19.call(RubyKernel$INVOKER$s$0$3$eval19.gen)
    at
    org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:180)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:70)
    at org.jruby.ast.FCallManyArgsNode.interpret(FCallManyArgsNode.java:60)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
    at
    org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:112)
    at
    org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:126)
    at
    org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:173)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:70)
    at org.jruby.ast.CallManyArgsNode.interpret(CallManyArgsNode.java:59)
    at org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
    at
    org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:225)
    at
    org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:214)
    at
    org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:202)
    at org.jruby.ast.CallTwoArgNode.interpret(CallTwoArgNode.java:59)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at org.jruby.ast.RescueNode.executeBody(RescueNode.java:221)
    at org.jruby.ast.RescueNode.interpret(RescueNode.java:116)
    at org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
    at
    org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)
    at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:157)
    at
    org.jruby.runtime.Interpreted19Block.yieldSpecific(Interpreted19Block.java:130)
    at org.jruby.runtime.Block.yieldSpecific(Block.java:111)
    at org.jruby.ast.ZYieldNode.interpret(ZYieldNode.java:25)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.EnsureNode.interpret(EnsureNode.java:96)
    at org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
    at
    org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:204)
    at
    org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:206)
    at
    org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:177)
    at
    org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:188)
    at
    org.jruby.ast.FCallOneArgBlockNode.interpret(FCallOneArgBlockNode.java:34)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
    at
    org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)
    at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:194)
    at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:177)
    at
    org.jruby.runtime.Interpreted19Block.yieldSpecific(Interpreted19Block.java:140)
    at org.jruby.runtime.Block.yieldSpecific(Block.java:129)
    at org.jruby.ast.YieldTwoNode.interpret(YieldTwoNode.java:31)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at org.jruby.ast.IfNode.interpret(IfNode.java:116)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at org.jruby.ast.RescueNode.executeBody(RescueNode.java:221)
    at org.jruby.ast.RescueNode.interpret(RescueNode.java:116)
    at org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
    at
    org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)
    at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:157)
    at
    org.jruby.runtime.Interpreted19Block.yieldSpecific(Interpreted19Block.java:130)
    at org.jruby.runtime.Block.yieldSpecific(Block.java:111)
    at org.jruby.RubyKernel.loop(RubyKernel.java:1501)
    at
    org.jruby.RubyKernel$INVOKER$s$0$0$loop.call(RubyKernel$INVOKER$s$0$0$loop.gen)
    at
    org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:316)
    at
    org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:145)
    at
    org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:154)
    at org.jruby.ast.FCallNoArgBlockNode.interpret(FCallNoArgBlockNode.java:32)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at
    org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
    at
    org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)
    at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:157)
    at org.jruby.runtime.Block.yield(Block.java:142)
    at org.jruby.RubyContinuation.enter(RubyContinuation.java:107)
    at org.jruby.RubyKernel.rbCatch19Common(RubyKernel.java:1271)
    at org.jruby.RubyKernel.rbCatch19(RubyKernel.java:1264)
    at
    org.jruby.RubyKernel$INVOKER$s$rbCatch19.call(RubyKernel$INVOKER$s$rbCatch19.gen)
    at
    org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:336)
    at
    org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:179)
    at
    org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:188)
    <span class="Apple-tab-s
    ...

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/637a43b9-8298-47d5-b35f-539ec03dfbd0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.