# StackOverflowError after node connect, java api
**URL:** https://discuss.elastic.co/t/stackoverflowerror-after-node-connect-java-api/3529
**Category:** Elasticsearch
**Created:** [November 6, 2010, 6:32pm UTC](https://discuss.elastic.co/t/stackoverflowerror-after-node-connect-java-api/3529 "2010-11-06T18:32:56Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@Sebastian\_Gavarini](https://discuss.elastic.co/u/Sebastian_Gavarini)
#### Post date: [November 6, 2010, 6:32pm UTC](https://discuss.elastic.co/t/stackoverflowerror-after-node-connect-java-api/3529/1 "2010-11-06T18:32:56Z")
Hi all,
I am using ES 0.12.1, java api, local gw, JRockit 1.6, and I am seeing
from time to time in my logs:
Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
thread-1" java.lang.StackOverflowError
Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
thread-2" java.lang.StackOverflowError
Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
thread-3" java.lang.StackOverflowError
Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
thread-4" java.lang.StackOverflowError
Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
thread-5" java.lang.StackOverflowError
Exception in thread "elasticsearch[Summers,
Alexander]clusterService#updateTask-pool-5-thread-1"
java.lang.StackOverflowError
This happens approximately one time every 20 runs of my client code.
It is happening before any of my actual work is started, just after
the cluster join is finished.
The code that joins the cluster in my client side is:
```
private Node buildNode() {
return
```
NodeBuilder.nodeBuilder().client(true).loadConfigSettings(true).node();
}
It's a singleton Guice injected object.
I am using a runtime parameter -Des.network.host=10.0.0.1 for the IP,
and the rest of the configuration is on the file elasticsearch.yml:
cluster:
name: test
transport:
tcp:
port: 5000-5099
connect\_timeout: 5s
http:
port: 5200-5299
discovery:
zen:
ping:
multicast:
group: 224.2.2.4
fd:
ping\_interval: 5s
ping\_timeout: 20s
ping\_retries: 3
threadpool:
scaling:
min: 2
max: 10
scheduled\_size: 6
gateway:
type: none
Does anybody sees an error that could be causing that behavior?
Thanks,
Sebastian.
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [November 6, 2010, 7:19pm UTC](https://discuss.elastic.co/t/stackoverflowerror-after-node-connect-java-api/3529/2 "2010-11-06T19:19:15Z")
Hi,
Strange... . Usually a stack overflow exception also includes the stack
trace up to where it happens. I can guess of several reason why this might
happen:
1. Using jrockit? Can you try and run it with openjdk / sun jdk?
2. elasticsearch jarjar guice, maybe the fact that Guice is also used messes
it up?
If you can get to the stack trace of why this happens, it would be much
simpler to analyze it. Maybe you can debug run it, and have an exception
breakpoint on StackOverflowException?
-shay.banon
On Sat, Nov 6, 2010 at 8:32 PM, Sebastian [sgavarini@gmail.com](mailto:sgavarini@gmail.com) wrote:
> Hi all,
> I am using ES 0.12.1, java api, local gw, JRockit 1.6, and I am seeing
> from time to time in my logs:
>
> Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> thread-1" java.lang.StackOverflowError
> Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> thread-2" java.lang.StackOverflowError
> Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> thread-3" java.lang.StackOverflowError
> Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> thread-4" java.lang.StackOverflowError
> Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> thread-5" java.lang.StackOverflowError
> Exception in thread "elasticsearch[Summers,
> Alexander]clusterService#updateTask-pool-5-thread-1"
> java.lang.StackOverflowError
>
> This happens approximately one time every 20 runs of my client code.
> It is happening before any of my actual work is started, just after
> the cluster join is finished.
> The code that joins the cluster in my client side is:
>
> ```
> private Node buildNode() {
> return
>
> ```
>
> NodeBuilder.nodeBuilder().client(true).loadConfigSettings(true).node();
> }
>
> It's a singleton Guice injected object.
> I am using a runtime parameter -Des.network.host=10.0.0.1 for the IP,
> and the rest of the configuration is on the file elasticsearch.yml:
>
> cluster:
> name: test
> transport:
> tcp:
> port: 5000-5099
> connect\_timeout: 5s
> http:
> port: 5200-5299
> discovery:
> zen:
> ping:
> multicast:
> group: 224.2.2.4
> fd:
> ping\_interval: 5s
> ping\_timeout: 20s
> ping\_retries: 3
> threadpool:
> scaling:
> min: 2
> max: 10
> scheduled\_size: 6
>
> gateway:
> type: none
>
> Does anybody sees an error that could be causing that behavior?
> Thanks,
> Sebastian.
---
### Author:  [@Sebastian\_Gavarini](https://discuss.elastic.co/u/Sebastian_Gavarini)
#### Post date: [November 6, 2010, 9:00pm UTC](https://discuss.elastic.co/t/stackoverflowerror-after-node-connect-java-api/3529/3 "2010-11-06T21:00:48Z")
Hi Shay,
Thanks for your quick answer. I did a test with debug run as you
suggested, I don't know what you mean by "exception breakpoint", but
when the exception fires the debugger shows the class:
ThreadPoolExecutor$Worker
public void run() {
try {
Runnable task = firstTask;
firstTask = null;
while (task != null || (task = getTask()) != null) {
runTask(task);
task = null;
}
} finally {
> > > > ```
> > > > workerDone(this);
> > > >
> > > > ```
```
}
}
```
line 911
Not very helpful, and all the values in the trace are null or empty,
except backtrace (again I don't know what is that):
StackOverflowError
backtrace = [139779368723408, 139779481643694, 139779446576800,
139779481643666, 139779471734800, 139779478653446, 139779368723408,
139779481643698, 139779446576800, 139779481643666, 139779471734800,
139779478653446, 139779368723408, 139779481643698, 139779446576800,
139779481643666, 139779471734800, 139779478653446, 139779368723408,
139779481643698, 139779446576800, 139779481643666, 139779471734800,
139779478653446, 139779368723408, 139779481643698, 139779446576800,
139779481643666, 139779471734800, 139779478653446, 139779368723408,
139779481643698, 139779446576800, 139779481643666, 139779471734800,
139779478653446, 139779368723408, 139779481643698, 139779446576800,
139779481643666, 139779471734800, 139779478653446, 139779368723408,
139779481643698, 139779446576800, 139779481643666, 139779471734800,
139779478653446, 139779368723408, 139779481643698, 139779446576800,
139779481643666, 139779471734800, 139779478653446, 139779368723408,
139779481643698, 139779446576800, 139779481643666, 139779471734800,
139779478653446, 139779368723408, 139779481643698, 139779446576800,
139779481643666]
The logs again are:
Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-1"
java.lang.StackOverflowError
Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-2"
java.lang.StackOverflowError
Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-3"
java.lang.StackOverflowError
Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-4"
java.lang.StackOverflowError
Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-5"
java.lang.StackOverflowError
Exception in thread "elasticsearch[Portal]clusterService#updateTask-
pool-5-thread-1" java.lang.StackOverflowError
I am seeing 5 threads again, but in my configuration I have 6 for
scheduled size, and 2-10 for min-max, any idea of where 5 comes from?
what's "tp" pool?
I had to run it like 10 times, but it eventually happened. I am
running Linux Ubuntu Maverick, standard JRockit configuration (no
stack memory messing around), in an AMD X2, (2 cores).
I will try changing JRockit to Sun's, any other idea is more than
welcomed. Nevertheless I am not inclined to think it's the cause, it's
more likely some classloading, constructors, or something like that
race condition.
Regarding Guice, I can't get rid of it that easily, maybe I could just
use your embedded Guice in elasticsearch (removing google's import in
favor of yours), for a quick test if you think it's useful.
Thanks,
Sebastian.
On Nov 6, 4:19 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> Hi,
>
> Strange... . Usually a stack overflow exception also includes the stack
> trace up to where it happens. I can guess of several reason why this might
> happen:
>
> 1. Using jrockit? Can you try and run it with openjdk / sun jdk?
> 2. elasticsearch jarjar guice, maybe the fact that Guice is also used messes
> it up?
>
> If you can get to the stack trace of why this happens, it would be much
> simpler to analyze it. Maybe you can debug run it, and have an exception
> breakpoint on StackOverflowException?
>
> -shay.banon
>
> On Sat, Nov 6, 2010 at 8:32 PM, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com) wrote:
>
> > Hi all,
> > I am using ES 0.12.1, java api, local gw, JRockit 1.6, and I am seeing
> > from time to time in my logs:
>
> > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > thread-1" java.lang.StackOverflowError
> > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > thread-2" java.lang.StackOverflowError
> > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > thread-3" java.lang.StackOverflowError
> > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > thread-4" java.lang.StackOverflowError
> > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > thread-5" java.lang.StackOverflowError
> > Exception in thread "elasticsearch[Summers,
> > Alexander]clusterService#updateTask-pool-5-thread-1"
> > java.lang.StackOverflowError
>
> > This happens approximately one time every 20 runs of my client code.
> > It is happening before any of my actual work is started, just after
> > the cluster join is finished.
> > The code that joins the cluster in my client side is:
>
> > ```
> > private Node buildNode() {
> > return
> >
> > ```
> >
> > NodeBuilder.nodeBuilder().client(true).loadConfigSettings(true).node();
> > }
>
> > It's a singleton Guice injected object.
> > I am using a runtime parameter -Des.network.host=10.0.0.1 for the IP,
> > and the rest of the configuration is on the file elasticsearch.yml:
>
> > cluster:
> > name: test
> > transport:
> > tcp:
> > port: 5000-5099
> > connect\_timeout: 5s
> > http:
> > port: 5200-5299
> > discovery:
> > zen:
> > ping:
> > multicast:
> > group: 224.2.2.4
> > fd:
> > ping\_interval: 5s
> > ping\_timeout: 20s
> > ping\_retries: 3
> > threadpool:
> > scaling:
> > min: 2
> > max: 10
> > scheduled\_size: 6
>
> > gateway:
> > type: none
>
> > Does anybody sees an error that could be causing that behavior?
> > Thanks,
> > Sebastian.
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [November 6, 2010, 9:22pm UTC](https://discuss.elastic.co/t/stackoverflowerror-after-node-connect-java-api/3529/4 "2010-11-06T21:22:25Z")
Hi,
An exception breakpoint allows, using the IDE, to break when specific
exception occurs. In IntelliJ, you go over to the Run Menue, click the
Breakpoints option, and the second tab there are exception breakpoints. You
can simply add one for the StackOverflow exception. Then, the importnat bit
here is the debug stack trace, and which methods keep repeating.
If you can try the sun jdk it would be great. Never tried to debug a Stack
Overflow in JRockit, but the ones I saw on the sun jdk do show the stack
trace, and then its easy to understand where it is coming from, as you see
the method(s) that keep repeating.
The [tp] thread pool is the main thread pool in elasticsearch.
-shay.banon
On Sat, Nov 6, 2010 at 11:00 PM, Sebastian [sgavarini@gmail.com](mailto:sgavarini@gmail.com) wrote:
> Hi Shay,
>
> Thanks for your quick answer. I did a test with debug run as you
> suggested, I don't know what you mean by "exception breakpoint", but
> when the exception fires the debugger shows the class:
>
> ThreadPoolExecutor$Worker
> public void run() {
> try {
> Runnable task = firstTask;
> firstTask = null;
> while (task != null || (task = getTask()) != null) {
> runTask(task);
> task = null;
> }
> } finally {
>
> > > > > ```
> > > > > workerDone(this);
> > > > >
> > > > > ```
>
> ```
> }
> }
>
> ```
>
> line 911
>
> Not very helpful, and all the values in the trace are null or empty,
> except backtrace (again I don't know what is that):
> StackOverflowError
> backtrace = [139779368723408, 139779481643694, 139779446576800,
> 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> 139779481643666]
>
> The logs again are:
> Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-1"
> java.lang.StackOverflowError
> Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-2"
> java.lang.StackOverflowError
> Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-3"
> java.lang.StackOverflowError
> Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-4"
> java.lang.StackOverflowError
> Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-5"
> java.lang.StackOverflowError
> Exception in thread "elasticsearch[Portal]clusterService#updateTask-
> pool-5-thread-1" java.lang.StackOverflowError
>
> I am seeing 5 threads again, but in my configuration I have 6 for
> scheduled size, and 2-10 for min-max, any idea of where 5 comes from?
> what's "tp" pool?
>
> I had to run it like 10 times, but it eventually happened. I am
> running Linux Ubuntu Maverick, standard JRockit configuration (no
> stack memory messing around), in an AMD X2, (2 cores).
>
> I will try changing JRockit to Sun's, any other idea is more than
> welcomed. Nevertheless I am not inclined to think it's the cause, it's
> more likely some classloading, constructors, or something like that
> race condition.
>
> Regarding Guice, I can't get rid of it that easily, maybe I could just
> use your embedded Guice in elasticsearch (removing google's import in
> favor of yours), for a quick test if you think it's useful.
>
> Thanks,
> Sebastian.
>
> On Nov 6, 4:19 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
>
> > Hi,
> >
> > Strange... . Usually a stack overflow exception also includes the stack
> > trace up to where it happens. I can guess of several reason why this
> > might
> > happen:
> >
> > 1. Using jrockit? Can you try and run it with openjdk / sun jdk?
> > 2. elasticsearch jarjar guice, maybe the fact that Guice is also used
> > messes
> > it up?
> >
> > If you can get to the stack trace of why this happens, it would be much
> > simpler to analyze it. Maybe you can debug run it, and have an exception
> > breakpoint on StackOverflowException?
> >
> > -shay.banon
> >
> > On Sat, Nov 6, 2010 at 8:32 PM, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com) wrote:
> >
> > > Hi all,
> > > I am using ES 0.12.1, java api, local gw, JRockit 1.6, and I am seeing
> > > from time to time in my logs:
> >
> > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > thread-1" java.lang.StackOverflowError
> > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > thread-2" java.lang.StackOverflowError
> > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > thread-3" java.lang.StackOverflowError
> > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > thread-4" java.lang.StackOverflowError
> > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > thread-5" java.lang.StackOverflowError
> > > Exception in thread "elasticsearch[Summers,
> > > Alexander]clusterService#updateTask-pool-5-thread-1"
> > > java.lang.StackOverflowError
> >
> > > This happens approximately one time every 20 runs of my client code.
> > > It is happening before any of my actual work is started, just after
> > > the cluster join is finished.
> > > The code that joins the cluster in my client side is:
> >
> > > ```
> > > private Node buildNode() {
> > > return
> > >
> > > ```
> > >
> > > NodeBuilder.nodeBuilder().client(true).loadConfigSettings(true).node();
> > > }
> >
> > > It's a singleton Guice injected object.
> > > I am using a runtime parameter -Des.network.host=10.0.0.1 for the IP,
> > > and the rest of the configuration is on the file elasticsearch.yml:
> >
> > > cluster:
> > > name: test
> > > transport:
> > > tcp:
> > > port: 5000-5099
> > > connect\_timeout: 5s
> > > http:
> > > port: 5200-5299
> > > discovery:
> > > zen:
> > > ping:
> > > multicast:
> > > group: 224.2.2.4
> > > fd:
> > > ping\_interval: 5s
> > > ping\_timeout: 20s
> > > ping\_retries: 3
> > > threadpool:
> > > scaling:
> > > min: 2
> > > max: 10
> > > scheduled\_size: 6
> >
> > > gateway:
> > > type: none
> >
> > > Does anybody sees an error that could be causing that behavior?
> > > Thanks,
> > > Sebastian.
---
### Author:  [@Sebastian\_Gavarini](https://discuss.elastic.co/u/Sebastian_Gavarini)
#### Post date: [November 6, 2010, 11:27pm UTC](https://discuss.elastic.co/t/stackoverflowerror-after-node-connect-java-api/3529/5 "2010-11-06T23:27:08Z")
Hi Shay,
I tried with open jdk 1.6 and I didn't see the error. Back with
JRockit the exception breakpoint didn't work. Then I did another
simple test, and even though I didn't see the exact same error as
before I think I might have a problem with slf4j.
When trying to run a simple test just joining the cluster from ten
threads:
On Eclipse I see
SLF4J: The following loggers will not work becasue they were created
SLF4J: during the default configuration phase of the underlying
logging system.
SLF4J: See also [SLF4J Error Codes](http://www.slf4j.org/codes.html#substituteLogger)
SLF4J: node
SLF4J: node
SLF4J: node
SLF4J: node
On ES server logs I see a couple of:
[2010-11-06 20:23:06,264][INFO][cluster.service] [Algrim
the Strong] removed {[Genis-Vell][i0Y0QXoDRbOrDVr2fNFvnw][inet[/
10.0.0.1:5005]]{client=true, data=false},}, reason: zen-disco-
node\_failed([Genis-Vell][i0Y0QXoDRbOrDVr2fNFvnw][inet[/10.0.0.1:5005]]
{client=true, data=false}), reason transport disconnected
Exception in thread "elasticsearch[Algrim the
Strong]clusterService#updateTask-pool-5-thread-14"
java.lang.StackOverflowError
Exception in thread "elasticsearch[Algrim the
Strong]clusterService#updateTask-pool-5-thread-15"
java.lang.StackOverflowError
If I remove slf4j from my poms I see on Eclipse:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: See [SLF4J Error Codes](http://www.slf4j.org/codes.html#StaticLoggerBinder) for
further details.
java.lang.NoClassDefFoundError: org/slf4j/impl/StaticLoggerBinder
at org.slf4j.LoggerFactory.staticInitialize(LoggerFactory.java:83)
at org.slf4j.LoggerFactory.(LoggerFactory.java:73)
at
org.elasticsearch.common.logging.slf4j.Slf4jESLoggerFactory.newInstance(Slf4jESLoggerFactory.java:
32)
at
org.elasticsearch.common.logging.ESLoggerFactory.getLogger(ESLoggerFactory.java:
60)
at org.elasticsearch.common.logging.Loggers.getLogger(Loggers.java:
142)
at org.elasticsearch.common.logging.Loggers.getLogger(Loggers.java:
121)
at
org.elasticsearch.node.internal.InternalNode.(InternalNode.java:
111)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:159)
at org.elasticsearch.node.NodeBuilder.node(NodeBuilder.java:166)
I am not sure if this is related but it could, I never used slf4j
before, but read that by adding:
org.slf4j
slf4j-log4j12
true
to my project it should log to log4j.
Am I messing things up? could you please tell me what's the expected
configuration for a java client api?
Thanks,
Sebastian.
On Nov 6, 6:22 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> Hi,
>
> An exception breakpoint allows, using the IDE, to break when specific
> exception occurs. In IntelliJ, you go over to the Run Menue, click the
> Breakpoints option, and the second tab there are exception breakpoints. You
> can simply add one for the StackOverflow exception. Then, the importnat bit
> here is the debug stack trace, and which methods keep repeating.
>
> If you can try the sun jdk it would be great. Never tried to debug a Stack
> Overflow in JRockit, but the ones I saw on the sun jdk do show the stack
> trace, and then its easy to understand where it is coming from, as you see
> the method(s) that keep repeating.
>
> The [tp] thread pool is the main thread pool in elasticsearch.
>
> -shay.banon
>
> On Sat, Nov 6, 2010 at 11:00 PM, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com) wrote:
>
> > Hi Shay,
>
> > Thanks for your quick answer. I did a test with debug run as you
> > suggested, I don't know what you mean by "exception breakpoint", but
> > when the exception fires the debugger shows the class:
>
> > ThreadPoolExecutor$Worker
> > public void run() {
> > try {
> > Runnable task = firstTask;
> > firstTask = null;
> > while (task != null || (task = getTask()) != null) {
> > runTask(task);
> > task = null;
> > }
> > } finally {
> >
> > > > > > ```
> > > > > > workerDone(this);
> > > > > >
> > > > > > ```
> >
> > ```
> > }
> > }
> >
> > ```
> >
> > line 911
>
> > Not very helpful, and all the values in the trace are null or empty,
> > except backtrace (again I don't know what is that):
> > StackOverflowError
> > backtrace = [139779368723408, 139779481643694, 139779446576800,
> > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > 139779481643666]
>
> > The logs again are:
> > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-1"
> > java.lang.StackOverflowError
> > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-2"
> > java.lang.StackOverflowError
> > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-3"
> > java.lang.StackOverflowError
> > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-4"
> > java.lang.StackOverflowError
> > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-5"
> > java.lang.StackOverflowError
> > Exception in thread "elasticsearch[Portal]clusterService#updateTask-
> > pool-5-thread-1" java.lang.StackOverflowError
>
> > I am seeing 5 threads again, but in my configuration I have 6 for
> > scheduled size, and 2-10 for min-max, any idea of where 5 comes from?
> > what's "tp" pool?
>
> > I had to run it like 10 times, but it eventually happened. I am
> > running Linux Ubuntu Maverick, standard JRockit configuration (no
> > stack memory messing around), in an AMD X2, (2 cores).
>
> > I will try changing JRockit to Sun's, any other idea is more than
> > welcomed. Nevertheless I am not inclined to think it's the cause, it's
> > more likely some classloading, constructors, or something like that
> > race condition.
>
> > Regarding Guice, I can't get rid of it that easily, maybe I could just
> > use your embedded Guice in elasticsearch (removing google's import in
> > favor of yours), for a quick test if you think it's useful.
>
> > Thanks,
> > Sebastian.
>
> > On Nov 6, 4:19 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> >
> > > Hi,
>
> > > Strange... . Usually a stack overflow exception also includes the stack
> > > trace up to where it happens. I can guess of several reason why this
> > > might
> > > happen:
>
> > > 1. Using jrockit? Can you try and run it with openjdk / sun jdk?
> > > 2. elasticsearch jarjar guice, maybe the fact that Guice is also used
> > > messes
> > > it up?
>
> > > If you can get to the stack trace of why this happens, it would be much
> > > simpler to analyze it. Maybe you can debug run it, and have an exception
> > > breakpoint on StackOverflowException?
>
> > > -shay.banon
>
> > > On Sat, Nov 6, 2010 at 8:32 PM, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com) wrote:
> > >
> > > > Hi all,
> > > > I am using ES 0.12.1, java api, local gw, JRockit 1.6, and I am seeing
> > > > from time to time in my logs:
>
> > > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > > thread-1" java.lang.StackOverflowError
> > > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > > thread-2" java.lang.StackOverflowError
> > > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > > thread-3" java.lang.StackOverflowError
> > > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > > thread-4" java.lang.StackOverflowError
> > > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > > thread-5" java.lang.StackOverflowError
> > > > Exception in thread "elasticsearch[Summers,
> > > > Alexander]clusterService#updateTask-pool-5-thread-1"
> > > > java.lang.StackOverflowError
>
> > > > This happens approximately one time every 20 runs of my client code.
> > > > It is happening before any of my actual work is started, just after
> > > > the cluster join is finished.
> > > > The code that joins the cluster in my client side is:
>
> > > > ```
> > > > private Node buildNode() {
> > > > return
> > > >
> > > > ```
> > > >
> > > > NodeBuilder.nodeBuilder().client(true).loadConfigSettings(true).node();
> > > > }
>
> > > > It's a singleton Guice injected object.
> > > > I am using a runtime parameter -Des.network.host=10.0.0.1 for the IP,
> > > > and the rest of the configuration is on the file elasticsearch.yml:
>
> > > > cluster:
> > > > name: test
> > > > transport:
> > > > tcp:
> > > > port: 5000-5099
> > > > connect\_timeout: 5s
> > > > http:
> > > > port: 5200-5299
> > > > discovery:
> > > > zen:
> > > > ping:
> > > > multicast:
> > > > group: 224.2.2.4
> > > > fd:
> > > > ping\_interval: 5s
> > > > ping\_timeout: 20s
> > > > ping\_retries: 3
> > > > threadpool:
> > > > scaling:
> > > > min: 2
> > > > max: 10
> > > > scheduled\_size: 6
>
> > > > gateway:
> > > > type: none
>
> > > > Does anybody sees an error that could be causing that behavior?
> > > > Thanks,
> > > > Sebastian.
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [November 6, 2010, 11:59pm UTC](https://discuss.elastic.co/t/stackoverflowerror-after-node-connect-java-api/3529/6 "2010-11-06T23:59:56Z")
elasticsearch does not have a dependency on slf4j, are you adding it just
because of elasticsearch? Just add log4j to the classpath and see if it
happens (without slf4j, any of its jars), or don't add any logging jars, and
logging will be done using the JDK logging by default.
-shay.banon
On Sun, Nov 7, 2010 at 1:27 AM, Sebastian [sgavarini@gmail.com](mailto:sgavarini@gmail.com) wrote:
> Hi Shay,
>
> I tried with open jdk 1.6 and I didn't see the error. Back with
> JRockit the exception breakpoint didn't work. Then I did another
> simple test, and even though I didn't see the exact same error as
> before I think I might have a problem with slf4j.
>
> When trying to run a simple test just joining the cluster from ten
> threads:
>
> On Eclipse I see
>
> SLF4J: The following loggers will not work becasue they were created
> SLF4J: during the default configuration phase of the underlying
> logging system.
> SLF4J: See also [SLF4J Error Codes](http://www.slf4j.org/codes.html#substituteLogger)
> SLF4J: node
> SLF4J: node
> SLF4J: node
> SLF4J: node
>
> On ES server logs I see a couple of:
> [2010-11-06 20:23:06,264][INFO][cluster.service] [Algrim
> the Strong] removed {[Genis-Vell][i0Y0QXoDRbOrDVr2fNFvnw][inet[/
> 10.0.0.1:5005]]{client=true, data=false},}, reason: zen-disco-
> node\_failed([Genis-Vell][i0Y0QXoDRbOrDVr2fNFvnw][inet[/10.0.0.1:5005]]
> {client=true, data=false}), reason transport disconnected
> Exception in thread "elasticsearch[Algrim the
> Strong]clusterService#updateTask-pool-5-thread-14"
> java.lang.StackOverflowError
> Exception in thread "elasticsearch[Algrim the
> Strong]clusterService#updateTask-pool-5-thread-15"
> java.lang.StackOverflowError
>
> If I remove slf4j from my poms I see on Eclipse:
>
> SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> SLF4J: See [SLF4J Error Codes](http://www.slf4j.org/codes.html#StaticLoggerBinder) for
> further details.
> java.lang.NoClassDefFoundError: org/slf4j/impl/StaticLoggerBinder
> at org.slf4j.LoggerFactory.staticInitialize(LoggerFactory.java:83)
> at org.slf4j.LoggerFactory.(LoggerFactory.java:73)
> at
>
> org.elasticsearch.common.logging.slf4j.Slf4jESLoggerFactory.newInstance(Slf4jESLoggerFactory.java:
> 32)
> at
>
> org.elasticsearch.common.logging.ESLoggerFactory.getLogger(ESLoggerFactory.java:
> 60)
> at org.elasticsearch.common.logging.Loggers.getLogger(Loggers.java:
> 142)
> at org.elasticsearch.common.logging.Loggers.getLogger(Loggers.java:
> 121)
> at
> org.elasticsearch.node.internal.InternalNode.(InternalNode.java:
> 111)
> at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:159)
> at org.elasticsearch.node.NodeBuilder.node(NodeBuilder.java:166)
>
> I am not sure if this is related but it could, I never used slf4j
> before, but read that by adding:
>
> org.slf4j
> slf4j-log4j12
> true
>
> to my project it should log to log4j.
> Am I messing things up? could you please tell me what's the expected
> configuration for a java client api?
> Thanks,
> Sebastian.
>
> On Nov 6, 6:22 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
>
> > Hi,
> >
> > An exception breakpoint allows, using the IDE, to break when specific
> > exception occurs. In IntelliJ, you go over to the Run Menue, click the
> > Breakpoints option, and the second tab there are exception breakpoints.
> > You
> > can simply add one for the StackOverflow exception. Then, the importnat
> > bit
> > here is the debug stack trace, and which methods keep repeating.
> >
> > If you can try the sun jdk it would be great. Never tried to debug a
> > Stack
> > Overflow in JRockit, but the ones I saw on the sun jdk do show the stack
> > trace, and then its easy to understand where it is coming from, as you
> > see
> > the method(s) that keep repeating.
> >
> > The [tp] thread pool is the main thread pool in elasticsearch.
> >
> > -shay.banon
> >
> > On Sat, Nov 6, 2010 at 11:00 PM, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com) wrote:
> >
> > > Hi Shay,
> >
> > > Thanks for your quick answer. I did a test with debug run as you
> > > suggested, I don't know what you mean by "exception breakpoint", but
> > > when the exception fires the debugger shows the class:
> >
> > > ThreadPoolExecutor$Worker
> > > public void run() {
> > > try {
> > > Runnable task = firstTask;
> > > firstTask = null;
> > > while (task != null || (task = getTask()) != null) {
> > > runTask(task);
> > > task = null;
> > > }
> > > } finally {
> > >
> > > > > > > ```
> > > > > > > workerDone(this);
> > > > > > >
> > > > > > > ```
> > >
> > > ```
> > > }
> > > }
> > >
> > > ```
> > >
> > > line 911
> >
> > > Not very helpful, and all the values in the trace are null or empty,
> > > except backtrace (again I don't know what is that):
> > > StackOverflowError
> > > backtrace = [139779368723408, 139779481643694, 139779446576800,
> > > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > > 139779481643666]
> >
> > > The logs again are:
> > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-1"
> > > java.lang.StackOverflowError
> > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-2"
> > > java.lang.StackOverflowError
> > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-3"
> > > java.lang.StackOverflowError
> > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-4"
> > > java.lang.StackOverflowError
> > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-5"
> > > java.lang.StackOverflowError
> > > Exception in thread "elasticsearch[Portal]clusterService#updateTask-
> > > pool-5-thread-1" java.lang.StackOverflowError
> >
> > > I am seeing 5 threads again, but in my configuration I have 6 for
> > > scheduled size, and 2-10 for min-max, any idea of where 5 comes from?
> > > what's "tp" pool?
> >
> > > I had to run it like 10 times, but it eventually happened. I am
> > > running Linux Ubuntu Maverick, standard JRockit configuration (no
> > > stack memory messing around), in an AMD X2, (2 cores).
> >
> > > I will try changing JRockit to Sun's, any other idea is more than
> > > welcomed. Nevertheless I am not inclined to think it's the cause, it's
> > > more likely some classloading, constructors, or something like that
> > > race condition.
> >
> > > Regarding Guice, I can't get rid of it that easily, maybe I could just
> > > use your embedded Guice in elasticsearch (removing google's import in
> > > favor of yours), for a quick test if you think it's useful.
> >
> > > Thanks,
> > > Sebastian.
> >
> > > On Nov 6, 4:19 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> > >
> > > > Hi,
> >
> > > > Strange... . Usually a stack overflow exception also includes the
> > > > stack
> > > > trace up to where it happens. I can guess of several reason why this
> > > > might
> > > > happen:
> >
> > > > 1. Using jrockit? Can you try and run it with openjdk / sun jdk?
> > > > 2. elasticsearch jarjar guice, maybe the fact that Guice is also used
> > > > messes
> > > > it up?
> >
> > > > If you can get to the stack trace of why this happens, it would be
> > > > much
> > > > simpler to analyze it. Maybe you can debug run it, and have an
> > > > exception
> > > > breakpoint on StackOverflowException?
> >
> > > > -shay.banon
> >
> > > > On Sat, Nov 6, 2010 at 8:32 PM, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com)
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > > I am using ES 0.12.1, java api, local gw, JRockit 1.6, and I am
> > > > > seeing
> > > > > from time to time in my logs:
> >
> > > > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > > > thread-1" java.lang.StackOverflowError
> > > > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > > > thread-2" java.lang.StackOverflowError
> > > > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > > > thread-3" java.lang.StackOverflowError
> > > > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > > > thread-4" java.lang.StackOverflowError
> > > > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > > > thread-5" java.lang.StackOverflowError
> > > > > Exception in thread "elasticsearch[Summers,
> > > > > Alexander]clusterService#updateTask-pool-5-thread-1"
> > > > > java.lang.StackOverflowError
> >
> > > > > This happens approximately one time every 20 runs of my client
> > > > > code.
> > > > > It is happening before any of my actual work is started, just after
> > > > > the cluster join is finished.
> > > > > The code that joins the cluster in my client side is:
> >
> > > > > ```
> > > > > private Node buildNode() {
> > > > > return
> > > > >
> > > > > ```
>
> NodeBuilder.nodeBuilder().client(true).loadConfigSettings(true).node();
>
> > > > > ```
> > > > > }
> > > > >
> > > > > ```
> >
> > > > > It's a singleton Guice injected object.
> > > > > I am using a runtime parameter -Des.network.host=10.0.0.1 for the
> > > > > IP,
> > > > > and the rest of the configuration is on the file elasticsearch.yml:
> >
> > > > > cluster:
> > > > > name: test
> > > > > transport:
> > > > > tcp:
> > > > > port: 5000-5099
> > > > > connect\_timeout: 5s
> > > > > http:
> > > > > port: 5200-5299
> > > > > discovery:
> > > > > zen:
> > > > > ping:
> > > > > multicast:
> > > > > group: 224.2.2.4
> > > > > fd:
> > > > > ping\_interval: 5s
> > > > > ping\_timeout: 20s
> > > > > ping\_retries: 3
> > > > > threadpool:
> > > > > scaling:
> > > > > min: 2
> > > > > max: 10
> > > > > scheduled\_size: 6
> >
> > > > > gateway:
> > > > > type: none
> >
> > > > > Does anybody sees an error that could be causing that behavior?
> > > > > Thanks,
> > > > > Sebastian.
---
### Author:  [@Sebastian\_Gavarini](https://discuss.elastic.co/u/Sebastian_Gavarini)
#### Post date: [November 7, 2010, 12:39am UTC](https://discuss.elastic.co/t/stackoverflowerror-after-node-connect-java-api/3529/7 "2010-11-07T00:39:10Z")
You are right, I thought I needed it, I added a couple of new jars to
my project at the same time and one of them was using slf4j, but not
ES, I removed it and the slf4j messages went away.
Unfortunately not the stackOverflow. I connected the remote debugger
to my ES server, and I see a huge stack of:
SendRequestTransportException(RemoteTransportException.fillStack()
line 58
SendRequestTransportException.fillInStackTrace() line 34
...
Does that tells you something? because for me the chain goes up to
Throwable: public synchronized native Throwable fillInStackTrace();
Sebastian.
On Nov 6, 8:59 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> elasticsearch does not have a dependency on slf4j, are you adding it just
> because of elasticsearch? Just add log4j to the classpath and see if it
> happens (without slf4j, any of its jars), or don't add any logging jars, and
> logging will be done using the JDK logging by default.
>
> -shay.banon
>
> On Sun, Nov 7, 2010 at 1:27 AM, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com) wrote:
>
> > Hi Shay,
>
> > I tried with open jdk 1.6 and I didn't see the error. Back with
> > JRockit the exception breakpoint didn't work. Then I did another
> > simple test, and even though I didn't see the exact same error as
> > before I think I might have a problem with slf4j.
>
> > When trying to run a simple test just joining the cluster from ten
> > threads:
>
> > On Eclipse I see
>
> > SLF4J: The following loggers will not work becasue they were created
> > SLF4J: during the default configuration phase of the underlying
> > logging system.
> > SLF4J: See alsohttp://www.slf4j.org/codes.html#substituteLogger
> > SLF4J: node
> > SLF4J: node
> > SLF4J: node
> > SLF4J: node
>
> > On ES server logs I see a couple of:
> > [2010-11-06 20:23:06,264][INFO][cluster.service] [Algrim
> > the Strong] removed {[Genis-Vell][i0Y0QXoDRbOrDVr2fNFvnw][inet[/
> > 10.0.0.1:5005]]{client=true, data=false},}, reason: zen-disco-
> > node\_failed([Genis-Vell][i0Y0QXoDRbOrDVr2fNFvnw][inet[/10.0.0.1:5005]]
> > {client=true, data=false}), reason transport disconnected
> > Exception in thread "elasticsearch[Algrim the
> > Strong]clusterService#updateTask-pool-5-thread-14"
> > java.lang.StackOverflowError
> > Exception in thread "elasticsearch[Algrim the
> > Strong]clusterService#updateTask-pool-5-thread-15"
> > java.lang.StackOverflowError
>
> > If I remove slf4j from my poms I see on Eclipse:
>
> > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> > SLF4J: Seehttp://www.slf4j.org/codes.html#StaticLoggerBinderfor
> > further details.
> > java.lang.NoClassDefFoundError: org/slf4j/impl/StaticLoggerBinder
> > at org.slf4j.LoggerFactory.staticInitialize(LoggerFactory.java:83)
> > at org.slf4j.LoggerFactory.(LoggerFactory.java:73)
> > at
>
> > org.elasticsearch.common.logging.slf4j.Slf4jESLoggerFactory.newInstance(Slf 4jESLoggerFactory.java:
> > 32)
> > at
>
> > org.elasticsearch.common.logging.ESLoggerFactory.getLogger(ESLoggerFactory. java:
> > 60)
> > at org.elasticsearch.common.logging.Loggers.getLogger(Loggers.java:
> > 142)
> > at org.elasticsearch.common.logging.Loggers.getLogger(Loggers.java:
> > 121)
> > at
> > org.elasticsearch.node.internal.InternalNode.(InternalNode.java:
> > 111)
> > at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:159)
> > at org.elasticsearch.node.NodeBuilder.node(NodeBuilder.java:166)
>
> > I am not sure if this is related but it could, I never used slf4j
> > before, but read that by adding:
> >
> > org.slf4j
> > slf4j-log4j12
> > true
> >
> > to my project it should log to log4j.
> > Am I messing things up? could you please tell me what's the expected
> > configuration for a java client api?
> > Thanks,
> > Sebastian.
>
> > On Nov 6, 6:22 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> >
> > > Hi,
>
> > > An exception breakpoint allows, using the IDE, to break when specific
> > > exception occurs. In IntelliJ, you go over to the Run Menue, click the
> > > Breakpoints option, and the second tab there are exception breakpoints.
> > > You
> > > can simply add one for the StackOverflow exception. Then, the importnat
> > > bit
> > > here is the debug stack trace, and which methods keep repeating.
>
> > > If you can try the sun jdk it would be great. Never tried to debug a
> > > Stack
> > > Overflow in JRockit, but the ones I saw on the sun jdk do show the stack
> > > trace, and then its easy to understand where it is coming from, as you
> > > see
> > > the method(s) that keep repeating.
>
> > > The [tp] thread pool is the main thread pool in elasticsearch.
>
> > > -shay.banon
>
> > > On Sat, Nov 6, 2010 at 11:00 PM, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com) wrote:
> > >
> > > > Hi Shay,
>
> > > > Thanks for your quick answer. I did a test with debug run as you
> > > > suggested, I don't know what you mean by "exception breakpoint", but
> > > > when the exception fires the debugger shows the class:
>
> > > > ThreadPoolExecutor$Worker
> > > > public void run() {
> > > > try {
> > > > Runnable task = firstTask;
> > > > firstTask = null;
> > > > while (task != null || (task = getTask()) != null) {
> > > > runTask(task);
> > > > task = null;
> > > > }
> > > > } finally {
> > > >
> > > > > > > > ```
> > > > > > > > workerDone(this);
> > > > > > > >
> > > > > > > > ```
> > > >
> > > > ```
> > > > }
> > > > }
> > > >
> > > > ```
> > > >
> > > > line 911
>
> > > > Not very helpful, and all the values in the trace are null or empty,
> > > > except backtrace (again I don't know what is that):
> > > > StackOverflowError
> > > > backtrace = [139779368723408, 139779481643694, 139779446576800,
> > > > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > > > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > > > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > > > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > > > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > > > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > > > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > > > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > > > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > > > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > > > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > > > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > > > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > > > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > > > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > > > 139779481643666]
>
> > > > The logs again are:
> > > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-1"
> > > > java.lang.StackOverflowError
> > > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-2"
> > > > java.lang.StackOverflowError
> > > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-3"
> > > > java.lang.StackOverflowError
> > > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-4"
> > > > java.lang.StackOverflowError
> > > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-5"
> > > > java.lang.StackOverflowError
> > > > Exception in thread "elasticsearch[Portal]clusterService#updateTask-
> > > > pool-5-thread-1" java.lang.StackOverflowError
>
> > > > I am seeing 5 threads again, but in my configuration I have 6 for
> > > > scheduled size, and 2-10 for min-max, any idea of where 5 comes from?
> > > > what's "tp" pool?
>
> > > > I had to run it like 10 times, but it eventually happened. I am
> > > > running Linux Ubuntu Maverick, standard JRockit configuration (no
> > > > stack memory messing around), in an AMD X2, (2 cores).
>
> > > > I will try changing JRockit to Sun's, any other idea is more than
> > > > welcomed. Nevertheless I am not inclined to think it's the cause, it's
> > > > more likely some classloading, constructors, or something like that
> > > > race condition.
>
> > > > Regarding Guice, I can't get rid of it that easily, maybe I could just
> > > > use your embedded Guice in elasticsearch (removing google's import in
> > > > favor of yours), for a quick test if you think it's useful.
>
> > > > Thanks,
> > > > Sebastian.
>
> > > > On Nov 6, 4:19 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> > > >
> > > > > Hi,
>
> > > > > Strange... . Usually a stack overflow exception also includes the
> > > > > stack
> > > > > trace up to where it happens. I can guess of several reason why this
> > > > > might
> > > > > happen:
>
> > > > > 1. Using jrockit? Can you try and run it with openjdk / sun jdk?
> > > > > 2. elasticsearch jarjar guice, maybe the fact that Guice is also used
> > > > > messes
> > > > > it up?
>
> > > > > If you can get to the stack trace of why this happens, it would be
> > > > > much
> > > > > simpler to analyze it. Maybe you can debug run it, and have an
> > > > > exception
> > > > > breakpoint on StackOverflowException?
>
> > > > > -shay.banon
>
> > > > > On Sat, Nov 6, 2010 at 8:32 PM, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com)
> > > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > > I am using ES 0.12.1, java api, local gw, JRockit 1.6, and I am
> > > > > > seeing
> > > > > > from time to time in my logs:
>
> > > > > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > > > > thread-1" java.lang.StackOverflowError
> > > > > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > > > > thread-2" java.lang.StackOverflowError
> > > > > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > > > > thread-3" java.lang.StackOverflowError
> > > > > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > > > > thread-4" java.lang.StackOverflowError
> > > > > > Exception in thread "elasticsearch[Summers, Alexander][tp]-pool-1-
> > > > > > thread-5" java.lang.StackOverflowError
> > > > > > Exception in thread "elasticsearch[Summers,
> > > > > > Alexander]clusterService#updateTask-pool-5-thread-1"
> > > > > > java.lang.StackOverflowError
>
> > > > > > This happens approximately one time every 20 runs of my client
> > > > > > code.
> > > > > > It is happening before any of my actual work is started, just after
> > > > > > the cluster join is finished.
> > > > > > The code that joins the cluster in my client side is:
>
> > > > > > ```
> > > > > > private Node buildNode() {
> > > > > > return
> > > > > >
> > > > > > ```
>
> > NodeBuilder.nodeBuilder().client(true).loadConfigSettings(true).node();
> >
> > > > > > ```
> > > > > > }
> > > > > >
> > > > > > ```
>
> > > > > > It's a
>
> ...
>
> read more »
---
### Author:  [@Sebastian\_Gavarini](https://discuss.elastic.co/u/Sebastian_Gavarini)
#### Post date: [November 7, 2010, 12:54am UTC](https://discuss.elastic.co/t/stackoverflowerror-after-node-connect-java-api/3529/8 "2010-11-07T00:54:53Z")
I was able to get some more info with the remote debugger, my client
code runs two threads with:
Node node =
NodeBuilder.nodeBuilder().client(true).loadConfigSettings(true).node();
Client client = node.client();
System.out.println("got up to here");
client.close();
And I am seeing the "got up to here" messages.
The remote ES server stacks are:
Daemon Thread [elasticsearch[Sofen, Karla]clusterService#updateTask-
pool-5-thread-1] (Suspended (breakpoint at line 39 in
RemoteTransportException))
NodeNotConnectedException(RemoteTransportException).(String,
TransportAddress, String, Throwable) line: 39
NodeNotConnectedException(ConnectTransportException).(DiscoveryNode,
String, String, Throwable) line: 44
NodeNotConnectedException(ConnectTransportException).(DiscoveryNode,
String) line: 32
NodeNotConnectedException.(DiscoveryNode, String) line: 32
NettyTransport.nodeChannel(DiscoveryNode) line: 473
NettyTransport.sendRequest(DiscoveryNode, long, String, Streamable,
TransportRequestOptions) line: 391
TransportService.sendRequest(DiscoveryNode, String, Streamable,
TransportRequestOptions, TransportResponseHandler) line: 182
TransportService.sendRequest(DiscoveryNode, String, Streamable,
TransportResponseHandler) line: 170
PublishClusterStateAction.publish(ClusterState) line: 70
ZenDiscovery.publish(ClusterState) line: 221
DiscoveryService.publish(ClusterState) line: 110
InternalClusterService$2.run() line: 221
ThreadPoolExecutor$Worker.runTask(Runnable) line: 885
ThreadPoolExecutor$Worker.run() line: 907
Thread.run() line: 619
Daemon Thread [elasticsearch[Sofen, Karla]clusterService#updateTask-
pool-5-thread-1] (Suspended (breakpoint at line 58 in
RemoteTransportException))
SendRequestTransportException(RemoteTransportException).fillStack()
line: 58
SendRequestTransportException.fillInStackTrace() line: 34
SendRequestTransportException(RemoteTransportException).fillStack()
line: 58
SendRequestTransportException.fillInStackTrace() line: 34
SendRequestTransportException(Throwable).(String, Throwable)
line: 218
SendRequestTransportException(Exception).(String, Throwable)
line: 59
SendRequestTransportException(RuntimeException).(String,
Throwable) line: 61
SendRequestTransportException(ElasticSearchException).(String,
Throwable) line: 46
SendRequestTransportException(TransportException).(String,
Throwable) line: 34
SendRequestTransportException(RemoteTransportException).(String,
TransportAddress, String, Throwable) line: 39
SendRequestTransportException.(DiscoveryNode, String,
Throwable) line: 30
TransportService.sendRequest(DiscoveryNode, String, Streamable,
TransportRequestOptions, TransportResponseHandler) line: 197
TransportService.sendRequest(DiscoveryNode, String, Streamable,
TransportResponseHandler) line: 170
PublishClusterStateAction.publish(ClusterState) line: 70
ZenDiscovery.publish(ClusterState) line: 221
DiscoveryService.publish(ClusterState) line: 110
InternalClusterService$2.run() line: 221
ThreadPoolExecutor$Worker.runTask(Runnable) line: 885
ThreadPoolExecutor$Worker.run() line: 907
Thread.run() line: 619
I got the first one of them, hit continue a couple of times, and then
got the second one.
You can see the offending lines:
SendRequestTransportException(RemoteTransportException).fillStack()
line: 58
SendRequestTransportException.fillInStackTrace() line: 34
Do you want me to get some value from any variable?
On Nov 6, 9:39 pm, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com) wrote:
> You are right, I thought I needed it, I added a couple of new jars to
> my project at the same time and one of them was using slf4j, but not
> ES, I removed it and the slf4j messages went away.
> Unfortunately not the stackOverflow. I connected the remote debugger
> to my ES server, and I see a huge stack of:
>
> SendRequestTransportException(RemoteTransportException.fillStack()
> line 58
> SendRequestTransportException.fillInStackTrace() line 34
> ...
>
> Does that tells you something? because for me the chain goes up to
> Throwable: public synchronized native Throwable fillInStackTrace();
>
> Sebastian.
>
> On Nov 6, 8:59 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
>
> > elasticsearch does not have a dependency on slf4j, are you adding it just
> > because of elasticsearch? Just add log4j to the classpath and see if it
> > happens (without slf4j, any of its jars), or don't add any logging jars, and
> > logging will be done using the JDK logging by default.
>
> > -shay.banon
>
> > On Sun, Nov 7, 2010 at 1:27 AM, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com) wrote:
> >
> > > Hi Shay,
>
> > > I tried with open jdk 1.6 and I didn't see the error. Back with
> > > JRockit the exception breakpoint didn't work. Then I did another
> > > simple test, and even though I didn't see the exact same error as
> > > before I think I might have a problem with slf4j.
>
> > > When trying to run a simple test just joining the cluster from ten
> > > threads:
>
> > > On Eclipse I see
>
> > > SLF4J: The following loggers will not work becasue they were created
> > > SLF4J: during the default configuration phase of the underlying
> > > logging system.
> > > SLF4J: See alsohttp://www.slf4j.org/codes.html#substituteLogger
> > > SLF4J: node
> > > SLF4J: node
> > > SLF4J: node
> > > SLF4J: node
>
> > > On ES server logs I see a couple of:
> > > [2010-11-06 20:23:06,264][INFO][cluster.service] [Algrim
> > > the Strong] removed {[Genis-Vell][i0Y0QXoDRbOrDVr2fNFvnw][inet[/
> > > 10.0.0.1:5005]]{client=true, data=false},}, reason: zen-disco-
> > > node\_failed([Genis-Vell][i0Y0QXoDRbOrDVr2fNFvnw][inet[/10.0.0.1:5005]]
> > > {client=true, data=false}), reason transport disconnected
> > > Exception in thread "elasticsearch[Algrim the
> > > Strong]clusterService#updateTask-pool-5-thread-14"
> > > java.lang.StackOverflowError
> > > Exception in thread "elasticsearch[Algrim the
> > > Strong]clusterService#updateTask-pool-5-thread-15"
> > > java.lang.StackOverflowError
>
> > > If I remove slf4j from my poms I see on Eclipse:
>
> > > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> > > SLF4J: Seehttp://www.slf4j.org/codes.html#StaticLoggerBinderfor
> > > further details.
> > > java.lang.NoClassDefFoundError: org/slf4j/impl/StaticLoggerBinder
> > > at org.slf4j.LoggerFactory.staticInitialize(LoggerFactory.java:83)
> > > at org.slf4j.LoggerFactory.(LoggerFactory.java:73)
> > > at
>
> > > org.elasticsearch.common.logging.slf4j.Slf4jESLoggerFactory.newInstance(Slf 4jESLoggerFactory.java:
> > > 32)
> > > at
>
> > > org.elasticsearch.common.logging.ESLoggerFactory.getLogger(ESLoggerFactory. java:
> > > 60)
> > > at org.elasticsearch.common.logging.Loggers.getLogger(Loggers.java:
> > > 142)
> > > at org.elasticsearch.common.logging.Loggers.getLogger(Loggers.java:
> > > 121)
> > > at
> > > org.elasticsearch.node.internal.InternalNode.(InternalNode.java:
> > > 111)
> > > at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:159)
> > > at org.elasticsearch.node.NodeBuilder.node(NodeBuilder.java:166)
>
> > > I am not sure if this is related but it could, I never used slf4j
> > > before, but read that by adding:
> > >
> > > org.slf4j
> > > slf4j-log4j12
> > > true
> > >
> > > to my project it should log to log4j.
> > > Am I messing things up? could you please tell me what's the expected
> > > configuration for a java client api?
> > > Thanks,
> > > Sebastian.
>
> > > On Nov 6, 6:22 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> > >
> > > > Hi,
>
> > > > An exception breakpoint allows, using the IDE, to break when specific
> > > > exception occurs. In IntelliJ, you go over to the Run Menue, click the
> > > > Breakpoints option, and the second tab there are exception breakpoints.
> > > > You
> > > > can simply add one for the StackOverflow exception. Then, the importnat
> > > > bit
> > > > here is the debug stack trace, and which methods keep repeating.
>
> > > > If you can try the sun jdk it would be great. Never tried to debug a
> > > > Stack
> > > > Overflow in JRockit, but the ones I saw on the sun jdk do show the stack
> > > > trace, and then its easy to understand where it is coming from, as you
> > > > see
> > > > the method(s) that keep repeating.
>
> > > > The [tp] thread pool is the main thread pool in elasticsearch.
>
> > > > -shay.banon
>
> > > > On Sat, Nov 6, 2010 at 11:00 PM, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com) wrote:
> > > >
> > > > > Hi Shay,
>
> > > > > Thanks for your quick answer. I did a test with debug run as you
> > > > > suggested, I don't know what you mean by "exception breakpoint", but
> > > > > when the exception fires the debugger shows the class:
>
> > > > > ThreadPoolExecutor$Worker
> > > > > public void run() {
> > > > > try {
> > > > > Runnable task = firstTask;
> > > > > firstTask = null;
> > > > > while (task != null || (task = getTask()) != null) {
> > > > > runTask(task);
> > > > > task = null;
> > > > > }
> > > > > } finally {
> > > > >
> > > > > > > > > ```
> > > > > > > > > workerDone(this);
> > > > > > > > >
> > > > > > > > > ```
> > > > >
> > > > > ```
> > > > > }
> > > > > }
> > > > >
> > > > > ```
> > > > >
> > > > > line 911
>
> > > > > Not very helpful, and all the values in the trace are null or empty,
> > > > > except backtrace (again I don't know what is that):
> > > > > StackOverflowError
> > > > > backtrace = [139779368723408, 139779481643694, 139779446576800,
> > > > > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > > > > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > > > > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > > > > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > > > > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > > > > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > > > > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > > > > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > > > > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > > > > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > > > > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > > > > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > > > > 139779481643666, 139779471734800, 139779478653446, 139779368723408,
> > > > > 139779481643698, 139779446576800, 139779481643666, 139779471734800,
> > > > > 139779478653446, 139779368723408, 139779481643698, 139779446576800,
> > > > > 139779481643666]
>
> > > > > The logs again are:
> > > > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-1"
> > > > > java.lang.StackOverflowError
> > > > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-2"
> > > > > java.lang.StackOverflowError
> > > > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-3"
> > > > > java.lang.StackOverflowError
> > > > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-4"
> > > > > java.lang.StackOverflowError
> > > > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-5"
> > > > > java.lang.StackOverflowError
> > > > > Exception in thread "elasticsearch[Portal]clusterService#updateTask-
> > > > > pool-5-thread-1" java.lang.StackOverflowError
>
> > > > > I am seeing 5 threads again, but in my configuration I have 6 for
> > > > > scheduled size, and 2-10 for min-max, any idea of where 5 comes from?
> > > > > what's "tp" pool?
>
> > > > > I had to run it like 10 times, but it eventually happened. I am
> > > > > running Linux Ubuntu Maverick, standard JRockit configuration (no
> > > > > stack memory messing around), in an AMD X2, (2 cores).
>
> > > > > I will try changing JRockit to Sun's, any other idea is more than
> > > > > welcomed. Nevertheless I am not inclined to think it's the cause, it's
> > > > > more likely some classloading, constructors, or something like that
> > > > > race condition.
>
> > > > > Regarding Guice, I can't get rid of it that easily, maybe I could just
> > > > > use your embedded Guice in elasticsearch (removing google's import in
> > > > > favor of yours), for a quick test if you think it's useful.
>
> > > > > Thanks,
> > > > > Sebastian.
>
> > > > > On Nov 6, 4:19 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> > > > >
> > > > > > Hi,
>
> > > > > > Strange... . Usually a stack overflow exception also includes the
> > > > > > stack
> > > > > > trace up to where it happens. I can guess of several reason why this
> > > > > > might
> > > > > > happen:
>
> > > > > > 1. Using jrockit? Can you try and run it with openjdk / sun jdk?
> > > > > > 2. elasticsearch jarjar guice, maybe the fact that Guice is also used
> > > > > > messes
> > > > > > it up?
>
> > > > > > If you can get to the stack trace of why this happens, it would be
> > > > > > much
> > > > > > simpler to analyze it. Maybe you can debug run it, and have an
> > > > > > exception
> > > > > > breakpoint on StackOverflowException?
>
> > > > > > -shay.banon
>
> > > > > > On Sat, Nov 6, 2010 at 8:32 PM, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com)
> > > > > > wrote:
> > > > > >
> > > > > > > Hi all,
> > > > > > > I am using...
>
> read more »
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [November 7, 2010, 2:10am UTC](https://discuss.elastic.co/t/stackoverflowerror-after-node-connect-java-api/3529/9 "2010-11-07T02:10:29Z")
Hi,
Strange, seems like the (based on what you posted) that the stack
overflow comes from how filling stack trace is controlled by the transport
exceptions in elasticsearch. I have pushed better handling in this case,
lets see if it solves it. Its in master, so you will need to get
master/latest, build a distribution (simply run gradlew release, the
distribution will be under build/distributions). Remember to also add the
updated elasticsearch jar file to your client.
-shay.banon
On Sun, Nov 7, 2010 at 2:54 AM, Sebastian [sgavarini@gmail.com](mailto:sgavarini@gmail.com) wrote:
> I was able to get some more info with the remote debugger, my client
> code runs two threads with:
> Node node =
> NodeBuilder.nodeBuilder().client(true).loadConfigSettings(true).node();
> Client client = node.client();
> System.out.println("got up to here");
> client.close();
>
> And I am seeing the "got up to here" messages.
> The remote ES server stacks are:
>
> Daemon Thread [elasticsearch[Sofen, Karla]clusterService#updateTask-
> pool-5-thread-1] (Suspended (breakpoint at line 39 in
> RemoteTransportException))
> NodeNotConnectedException(RemoteTransportException).(String,
> TransportAddress, String, Throwable) line: 39
>
> NodeNotConnectedException(ConnectTransportException).(DiscoveryNode,
> String, String, Throwable) line: 44
>
> NodeNotConnectedException(ConnectTransportException).(DiscoveryNode,
> String) line: 32
> NodeNotConnectedException.(DiscoveryNode, String) line: 32
> NettyTransport.nodeChannel(DiscoveryNode) line: 473
> NettyTransport.sendRequest(DiscoveryNode, long, String, Streamable,
> TransportRequestOptions) line: 391
> TransportService.sendRequest(DiscoveryNode, String, Streamable,
> TransportRequestOptions, TransportResponseHandler) line: 182
> TransportService.sendRequest(DiscoveryNode, String, Streamable,
> TransportResponseHandler) line: 170
> PublishClusterStateAction.publish(ClusterState) line: 70
> ZenDiscovery.publish(ClusterState) line: 221
> DiscoveryService.publish(ClusterState) line: 110
> InternalClusterService$2.run() line: 221
> ThreadPoolExecutor$Worker.runTask(Runnable) line: 885
> ThreadPoolExecutor$Worker.run() line: 907
> Thread.run() line: 619
>
> Daemon Thread [elasticsearch[Sofen, Karla]clusterService#updateTask-
> pool-5-thread-1] (Suspended (breakpoint at line 58 in
> RemoteTransportException))
> SendRequestTransportException(RemoteTransportException).fillStack()
> line: 58
> SendRequestTransportException.fillInStackTrace() line: 34
> SendRequestTransportException(RemoteTransportException).fillStack()
> line: 58
> SendRequestTransportException.fillInStackTrace() line: 34
> SendRequestTransportException(Throwable).(String, Throwable)
> line: 218
> SendRequestTransportException(Exception).(String, Throwable)
> line: 59
> SendRequestTransportException(RuntimeException).(String,
> Throwable) line: 61
> SendRequestTransportException(ElasticSearchException).(String,
> Throwable) line: 46
> SendRequestTransportException(TransportException).(String,
> Throwable) line: 34
>
> SendRequestTransportException(RemoteTransportException).(String,
> TransportAddress, String, Throwable) line: 39
> SendRequestTransportException.(DiscoveryNode, String,
> Throwable) line: 30
> TransportService.sendRequest(DiscoveryNode, String, Streamable,
> TransportRequestOptions, TransportResponseHandler) line: 197
> TransportService.sendRequest(DiscoveryNode, String, Streamable,
> TransportResponseHandler) line: 170
> PublishClusterStateAction.publish(ClusterState) line: 70
> ZenDiscovery.publish(ClusterState) line: 221
> DiscoveryService.publish(ClusterState) line: 110
> InternalClusterService$2.run() line: 221
> ThreadPoolExecutor$Worker.runTask(Runnable) line: 885
> ThreadPoolExecutor$Worker.run() line: 907
> Thread.run() line: 619
>
> I got the first one of them, hit continue a couple of times, and then
> got the second one.
> You can see the offending lines:
> SendRequestTransportException(RemoteTransportException).fillStack()
> line: 58
> SendRequestTransportException.fillInStackTrace() line: 34
>
> Do you want me to get some value from any variable?
>
> On Nov 6, 9:39 pm, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com) wrote:
>
> > You are right, I thought I needed it, I added a couple of new jars to
> > my project at the same time and one of them was using slf4j, but not
> > ES, I removed it and the slf4j messages went away.
> > Unfortunately not the stackOverflow. I connected the remote debugger
> > to my ES server, and I see a huge stack of:
> >
> > SendRequestTransportException(RemoteTransportException.fillStack()
> > line 58
> > SendRequestTransportException.fillInStackTrace() line 34
> > ...
> >
> > Does that tells you something? because for me the chain goes up to
> > Throwable: public synchronized native Throwable fillInStackTrace();
> >
> > Sebastian.
> >
> > On Nov 6, 8:59 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> >
> > > elasticsearch does not have a dependency on slf4j, are you adding it
> > > just
> > > because of elasticsearch? Just add log4j to the classpath and see if it
> > > happens (without slf4j, any of its jars), or don't add any logging
> > > jars, and
> > > logging will be done using the JDK logging by default.
> >
> > > -shay.banon
> >
> > > On Sun, Nov 7, 2010 at 1:27 AM, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com) wrote:
> > >
> > > > Hi Shay,
> >
> > > > I tried with open jdk 1.6 and I didn't see the error. Back with
> > > > JRockit the exception breakpoint didn't work. Then I did another
> > > > simple test, and even though I didn't see the exact same error as
> > > > before I think I might have a problem with slf4j.
> >
> > > > When trying to run a simple test just joining the cluster from ten
> > > > threads:
> >
> > > > On Eclipse I see
> >
> > > > SLF4J: The following loggers will not work becasue they were created
> > > > SLF4J: during the default configuration phase of the underlying
> > > > logging system.
> > > > SLF4J: See alsohttp://www.slf4j.org/codes.html#substituteLogger
> > > > SLF4J: node
> > > > SLF4J: node
> > > > SLF4J: node
> > > > SLF4J: node
> >
> > > > On ES server logs I see a couple of:
> > > > [2010-11-06 20:23:06,264][INFO][cluster.service] [Algrim
> > > > the Strong] removed {[Genis-Vell][i0Y0QXoDRbOrDVr2fNFvnw][inet[/
> > > > 10.0.0.1:5005]]{client=true, data=false},}, reason: zen-disco-
> > > > node\_failed([Genis-Vell][i0Y0QXoDRbOrDVr2fNFvnw][inet[/10.0.0.1:5005
> > > > ]]
> > > > {client=true, data=false}), reason transport disconnected
> > > > Exception in thread "elasticsearch[Algrim the
> > > > Strong]clusterService#updateTask-pool-5-thread-14"
> > > > java.lang.StackOverflowError
> > > > Exception in thread "elasticsearch[Algrim the
> > > > Strong]clusterService#updateTask-pool-5-thread-15"
> > > > java.lang.StackOverflowError
> >
> > > > If I remove slf4j from my poms I see on Eclipse:
> >
> > > > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> > > > SLF4J: Seehttp://www.slf4j.org/codes.html#StaticLoggerBinderfor
> > > > further details.
> > > > java.lang.NoClassDefFoundError: org/slf4j/impl/StaticLoggerBinder
> > > > at
> > > > org.slf4j.LoggerFactory.staticInitialize(LoggerFactory.java:83)
> > > > at org.slf4j.LoggerFactory.(LoggerFactory.java:73)
> > > > at
> >
> > > >
>
> org.elasticsearch.common.logging.slf4j.Slf4jESLoggerFactory.newInstance(Slf
> 4jESLoggerFactory.java:
>
> > > > 1. at
> >
> > > >
>
> org.elasticsearch.common.logging.ESLoggerFactory.getLogger(ESLoggerFactory.
> java:
>
> > > > 1. at
> > > > org.elasticsearch.common.logging.Loggers.getLogger(Loggers.java:
> > > > 2. at
> > > > org.elasticsearch.common.logging.Loggers.getLogger(Loggers.java:
> > > > 3. at
>
> org.elasticsearch.node.internal.InternalNode.(InternalNode.java:
>
> > > > 1. at
> > > > org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:159)
> > > > at
> > > > org.elasticsearch.node.NodeBuilder.node(NodeBuilder.java:166)
> >
> > > > I am not sure if this is related but it could, I never used slf4j
> > > > before, but read that by adding:
> > > >
> > > > org.slf4j
> > > > slf4j-log4j12
> > > > true
> > > >
> > > > to my project it should log to log4j.
> > > > Am I messing things up? could you please tell me what's the expected
> > > > configuration for a java client api?
> > > > Thanks,
> > > > Sebastian.
> >
> > > > On Nov 6, 6:22 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> > > >
> > > > > Hi,
> >
> > > > > An exception breakpoint allows, using the IDE, to break when
> > > > > specific
> > > > > exception occurs. In IntelliJ, you go over to the Run Menue, click
> > > > > the
> > > > > Breakpoints option, and the second tab there are exception
> > > > > breakpoints.
> > > > > You
> > > > > can simply add one for the StackOverflow exception. Then, the
> > > > > importnat
> > > > > bit
> > > > > here is the debug stack trace, and which methods keep repeating.
> >
> > > > > If you can try the sun jdk it would be great. Never tried to
> > > > > debug a
> > > > > Stack
> > > > > Overflow in JRockit, but the ones I saw on the sun jdk do show the
> > > > > stack
> > > > > trace, and then its easy to understand where it is coming from, as
> > > > > you
> > > > > see
> > > > > the method(s) that keep repeating.
> >
> > > > > The [tp] thread pool is the main thread pool in elasticsearch.
> >
> > > > > -shay.banon
> >
> > > > > On Sat, Nov 6, 2010 at 11:00 PM, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com)
> > > > > wrote:
> > > > >
> > > > > > Hi Shay,
> >
> > > > > > Thanks for your quick answer. I did a test with debug run as you
> > > > > > suggested, I don't know what you mean by "exception breakpoint",
> > > > > > but
> > > > > > when the exception fires the debugger shows the class:
> >
> > > > > > ThreadPoolExecutor$Worker
> > > > > > public void run() {
> > > > > > try {
> > > > > > Runnable task = firstTask;
> > > > > > firstTask = null;
> > > > > > while (task != null || (task = getTask()) != null)
> > > > > > {
> > > > > > runTask(task);
> > > > > > task = null;
> > > > > > }
> > > > > > } finally {
> > > > > >
> > > > > > > > > > ```
> > > > > > > > > > workerDone(this);
> > > > > > > > > >
> > > > > > > > > > ```
> > > > > >
> > > > > > ```
> > > > > > }
> > > > > > }
> > > > > >
> > > > > > ```
> > > > > >
> > > > > > line 911
> >
> > > > > > Not very helpful, and all the values in the trace are null or
> > > > > > empty,
> > > > > > except backtrace (again I don't know what is that):
> > > > > > StackOverflowError
> > > > > > backtrace = [139779368723408, 139779481643694, 139779446576800,
> > > > > > 139779481643666, 139779471734800, 139779478653446,
> > > > > > 139779368723408,
> > > > > > 139779481643698, 139779446576800, 139779481643666,
> > > > > > 139779471734800,
> > > > > > 139779478653446, 139779368723408, 139779481643698,
> > > > > > 139779446576800,
> > > > > > 139779481643666, 139779471734800, 139779478653446,
> > > > > > 139779368723408,
> > > > > > 139779481643698, 139779446576800, 139779481643666,
> > > > > > 139779471734800,
> > > > > > 139779478653446, 139779368723408, 139779481643698,
> > > > > > 139779446576800,
> > > > > > 139779481643666, 139779471734800, 139779478653446,
> > > > > > 139779368723408,
> > > > > > 139779481643698, 139779446576800, 139779481643666,
> > > > > > 139779471734800,
> > > > > > 139779478653446, 139779368723408, 139779481643698,
> > > > > > 139779446576800,
> > > > > > 139779481643666, 139779471734800, 139779478653446,
> > > > > > 139779368723408,
> > > > > > 139779481643698, 139779446576800, 139779481643666,
> > > > > > 139779471734800,
> > > > > > 139779478653446, 139779368723408, 139779481643698,
> > > > > > 139779446576800,
> > > > > > 139779481643666, 139779471734800, 139779478653446,
> > > > > > 139779368723408,
> > > > > > 139779481643698, 139779446576800, 139779481643666,
> > > > > > 139779471734800,
> > > > > > 139779478653446, 139779368723408, 139779481643698,
> > > > > > 139779446576800,
> > > > > > 139779481643666]
> >
> > > > > > The logs again are:
> > > > > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-1"
> > > > > > java.lang.StackOverflowError
> > > > > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-2"
> > > > > > java.lang.StackOverflowError
> > > > > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-3"
> > > > > > java.lang.StackOverflowError
> > > > > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-4"
> > > > > > java.lang.StackOverflowError
> > > > > > Exception in thread "elasticsearch[Portal][tp]-pool-1-thread-5"
> > > > > > java.lang.StackOverflowError
> > > > > > Exception in thread
> > > > > > "elasticsearch[Portal]clusterService#updateTask-
> > > > > > pool-5-thread-1" java.lang.StackOverflowError
> >
> > > > > > I am seeing 5 threads again, but in my configuration I have 6 for
> > > > > > scheduled size, and 2-10 for min-max, any idea of where 5 comes
> > > > > > from?
> > > > > > what's "tp" pool?
> >
> > > > > > I had to run it like 10 times, but it eventually happened. I am
> > > > > > running Linux Ubuntu Maverick, standard JRockit configuration (no
> > > > > > stack memory messing around), in an AMD X2, (2 cores).
> >
> > > > > > I will try changing JRockit to Sun's, any other idea is more than
> > > > > > welcomed. Nevertheless I am not inclined to think it's the cause,
> > > > > > it's
> > > > > > more likely some classloading, constructors, or something like
> > > > > > that
> > > > > > race condition.
> >
> > > > > > Regarding Guice, I can't get rid of it that easily, maybe I could
> > > > > > just
> > > > > > use your embedded Guice in elasticsearch (removing google's
> > > > > > import in
> > > > > > favor of yours), for a quick test if you think it's useful.
> >
> > > > > > Thanks,
> > > > > > Sebastian.
> >
> > > > > > On Nov 6, 4:19 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com)
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> >
> > > > > > > Strange... . Usually a stack overflow exception also includes
> > > > > > > the
> > > > > > > stack
> > > > > > > trace up to where it happens. I can guess of several reason why
> > > > > > > this
> > > > > > > might
> > > > > > > happen:
> >
> > > > > > > 1. Using jrockit? Can you try and run it with openjdk / sun
> > > > > > > jdk?
> > > > > > > 2. elasticsearch jarjar guice, maybe the fact that Guice is
> > > > > > > also used
> > > > > > > messes
> > > > > > > it up?
> >
> > > > > > > If you can get to the stack trace of why this happens, it
> > > > > > > would be
> > > > > > > much
> > > > > > > simpler to analyze it. Maybe you can debug run it, and have an
> > > > > > > exception
> > > > > > > breakpoint on StackOverflowException?
> >
> > > > > > > -shay.banon
> >
> > > > > > > On Sat, Nov 6, 2010 at 8:32 PM, Sebastian \<[sgavar...@gmail.com](mailto:sgavar...@gmail.com)
> >
> > > > wrote:
> > > >
> > > > > > > > Hi all,
> > > > > > > > I am using...
> >
> > read more »
---
### Author:  [@Sebastian\_Gavarini](https://discuss.elastic.co/u/Sebastian_Gavarini)
#### Post date: [November 7, 2010, 3:07am UTC](https://discuss.elastic.co/t/stackoverflowerror-after-node-connect-java-api/3529/10 "2010-11-07T03:07:24Z")
Hi Shay,
Yep, you nailed it!, I run my test many times against master, with 2
up to 20 threads in JRockit and now it's working fine.
Thanks a lot,
Sebastian.
On Nov 6, 11:10 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> Hi,
>
> Strange, seems like the (based on what you posted) that the stack
> overflow comes from how filling stack trace is controlled by the transport
> exceptions in elasticsearch. I have pushed better handling in this case,
> lets see if it solves it. Its in master, so you will need to get
> master/latest, build a distribution (simply run gradlew release, the
> distribution will be under build/distributions). Remember to also add the
> updated elasticsearch jar file to your client.
>
> -shay.banon
>
> On Sun, Nov 7, 2010 at 2:54 AM, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com) wrote:
>
> > I was able to get some more info with the remote debugger, my client
> > code runs two threads with:
> > Node node =
> > NodeBuilder.nodeBuilder().client(true).loadConfigSettings(true).node();
> > Client client = node.client();
> > System.out.println("got up to here");
> > client.close();
>
> > And I am seeing the "got up to here" messages.
> > The remote ES server stacks are:
>
> > Daemon Thread [elasticsearch[Sofen, Karla]clusterService#updateTask-
> > pool-5-thread-1] (Suspended (breakpoint at line 39 in
> > RemoteTransportException))
> > NodeNotConnectedException(RemoteTransportException).(String,
> > TransportAddress, String, Throwable) line: 39
>
> > NodeNotConnectedException(ConnectTransportException).(DiscoveryNode,
> > String, String, Throwable) line: 44
>
> > NodeNotConnectedException(ConnectTransportException).(DiscoveryNode,
> > String) line: 32
> > NodeNotConnectedException.(DiscoveryNode, String) line: 32
> > NettyTransport.nodeChannel(DiscoveryNode) line: 473
> > NettyTransport.sendRequest(DiscoveryNode, long, String, Streamable,
> > TransportRequestOptions) line: 391
> > TransportService.sendRequest(DiscoveryNode, String, Streamable,
> > TransportRequestOptions, TransportResponseHandler) line: 182
> > TransportService.sendRequest(DiscoveryNode, String, Streamable,
> > TransportResponseHandler) line: 170
> > PublishClusterStateAction.publish(ClusterState) line: 70
> > ZenDiscovery.publish(ClusterState) line: 221
> > DiscoveryService.publish(ClusterState) line: 110
> > InternalClusterService$2.run() line: 221
> > ThreadPoolExecutor$Worker.runTask(Runnable) line: 885
> > ThreadPoolExecutor$Worker.run() line: 907
> > Thread.run() line: 619
>
> > Daemon Thread [elasticsearch[Sofen, Karla]clusterService#updateTask-
> > pool-5-thread-1] (Suspended (breakpoint at line 58 in
> > RemoteTransportException))
> > SendRequestTransportException(RemoteTransportException).fillStack()
> > line: 58
> > SendRequestTransportException.fillInStackTrace() line: 34
> > SendRequestTransportException(RemoteTransportException).fillStack()
> > line: 58
> > SendRequestTransportException.fillInStackTrace() line: 34
> > SendRequestTransportException(Throwable).(String, Throwable)
> > line: 218
> > SendRequestTransportException(Exception).(String, Throwable)
> > line: 59
> > SendRequestTransportException(RuntimeException).(String,
> > Throwable) line: 61
> > SendRequestTransportException(ElasticSearchException).(String,
> > Throwable) line: 46
> > SendRequestTransportException(TransportException).(String,
> > Throwable) line: 34
>
> > SendRequestTransportException(RemoteTransportException).(String,
> > TransportAddress, String, Throwable) line: 39
> > SendRequestTransportException.(DiscoveryNode, String,
> > Throwable) line: 30
> > TransportService.sendRequest(DiscoveryNode, String, Streamable,
> > TransportRequestOptions, TransportResponseHandler) line: 197
> > TransportService.sendRequest(DiscoveryNode, String, Streamable,
> > TransportResponseHandler) line: 170
> > PublishClusterStateAction.publish(ClusterState) line: 70
> > ZenDiscovery.publish(ClusterState) line: 221
> > DiscoveryService.publish(ClusterState) line: 110
> > InternalClusterService$2.run() line: 221
> > ThreadPoolExecutor$Worker.runTask(Runnable) line: 885
> > ThreadPoolExecutor$Worker.run() line: 907
> > Thread.run() line: 619
>
> > I got the first one of them, hit continue a couple of times, and then
> > got the second one.
> > You can see the offending lines:
> > SendRequestTransportException(RemoteTransportException).fillStack()
> > line: 58
> > SendRequestTransportException.fillInStackTrace() line: 34
>
> > Do you want me to get some value from any variable?
>
> > On Nov 6, 9:39 pm, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com) wrote:
> >
> > > You are right, I thought I needed it, I added a couple of new jars to
> > > my project at the same time and one of them was using slf4j, but not
> > > ES, I removed it and the slf4j messages went away.
> > > Unfortunately not the stackOverflow. I connected the remote debugger
> > > to my ES server, and I see a huge stack of:
>
> > > SendRequestTransportException(RemoteTransportException.fillStack()
> > > line 58
> > > SendRequestTransportException.fillInStackTrace() line 34
> > > ...
>
> > > Does that tells you something? because for me the chain goes up to
> > > Throwable: public synchronized native Throwable fillInStackTrace();
>
> > > Sebastian.
>
> > > On Nov 6, 8:59 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
>
> > > > elasticsearch does not have a dependency on slf4j, are you adding it
> > > > just
> > > > because of elasticsearch? Just add log4j to the classpath and see if it
> > > > happens (without slf4j, any of its jars), or don't add any logging
> > > > jars, and
> > > > logging will be done using the JDK logging by default.
>
> > > > -shay.banon
>
> > > > On Sun, Nov 7, 2010 at 1:27 AM, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com) wrote:
> > > >
> > > > > Hi Shay,
>
> > > > > I tried with open jdk 1.6 and I didn't see the error. Back with
> > > > > JRockit the exception breakpoint didn't work. Then I did another
> > > > > simple test, and even though I didn't see the exact same error as
> > > > > before I think I might have a problem with slf4j.
>
> > > > > When trying to run a simple test just joining the cluster from ten
> > > > > threads:
>
> > > > > On Eclipse I see
>
> > > > > SLF4J: The following loggers will not work becasue they were created
> > > > > SLF4J: during the default configuration phase of the underlying
> > > > > logging system.
> > > > > SLF4J: See alsohttp://www.slf4j.org/codes.html#substituteLogger
> > > > > SLF4J: node
> > > > > SLF4J: node
> > > > > SLF4J: node
> > > > > SLF4J: node
>
> > > > > On ES server logs I see a couple of:
> > > > > [2010-11-06 20:23:06,264][INFO][cluster.service] [Algrim
> > > > > the Strong] removed {[Genis-Vell][i0Y0QXoDRbOrDVr2fNFvnw][inet[/
> > > > > 10.0.0.1:5005]]{client=true, data=false},}, reason: zen-disco-
> > > > > node\_failed([Genis-Vell][i0Y0QXoDRbOrDVr2fNFvnw][inet[/10.0.0.1:5005
> > > > > ]]
> > > > > {client=true, data=false}), reason transport disconnected
> > > > > Exception in thread "elasticsearch[Algrim the
> > > > > Strong]clusterService#updateTask-pool-5-thread-14"
> > > > > java.lang.StackOverflowError
> > > > > Exception in thread "elasticsearch[Algrim the
> > > > > Strong]clusterService#updateTask-pool-5-thread-15"
> > > > > java.lang.StackOverflowError
>
> > > > > If I remove slf4j from my poms I see on Eclipse:
>
> > > > > SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
> > > > > SLF4J: Seehttp://www.slf4j.org/codes.html#StaticLoggerBinderfor
> > > > > further details.
> > > > > java.lang.NoClassDefFoundError: org/slf4j/impl/StaticLoggerBinder
> > > > > at
> > > > > org.slf4j.LoggerFactory.staticInitialize(LoggerFactory.java:83)
> > > > > at org.slf4j.LoggerFactory.(LoggerFactory.java:73)
> > > > > at
>
> > org.elasticsearch.common.logging.slf4j.Slf4jESLoggerFactory.newInstance(Slf
> > 4jESLoggerFactory.java:
> >
> > > > > 1. at
>
> > org.elasticsearch.common.logging.ESLoggerFactory.getLogger(ESLoggerFactory.
> > java:
> >
> > > > > 1. at
> > > > > org.elasticsearch.common.logging.Loggers.getLogger(Loggers.java:
> > > > > 2. at
> > > > > org.elasticsearch.common.logging.Loggers.getLogger(Loggers.java:
> > > > > 3. at
>
> > org.elasticsearch.node.internal.InternalNode.(InternalNode.java:
> >
> > > > > 1. at
> > > > > org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:159)
> > > > > at
> > > > > org.elasticsearch.node.NodeBuilder.node(NodeBuilder.java:166)
>
> > > > > I am not sure if this is related but it could, I never used slf4j
> > > > > before, but read that by adding:
> > > > >
> > > > > org.slf4j
> > > > > slf4j-log4j12
> > > > > true
> > > > >
> > > > > to my project it should log to log4j.
> > > > > Am I messing things up? could you please tell me what's the expected
> > > > > configuration for a java client api?
> > > > > Thanks,
> > > > > Sebastian.
>
> > > > > On Nov 6, 6:22 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> > > > >
> > > > > > Hi,
>
> > > > > > An exception breakpoint allows, using the IDE, to break when
> > > > > > specific
> > > > > > exception occurs. In IntelliJ, you go over to the Run Menue, click
> > > > > > the
> > > > > > Breakpoints option, and the second tab there are exception
> > > > > > breakpoints.
> > > > > > You
> > > > > > can simply add one for the StackOverflow exception. Then, the
> > > > > > importnat
> > > > > > bit
> > > > > > here is the debug stack trace, and which methods keep repeating.
>
> > > > > > If you can try the sun jdk it would be great. Never tried to
> > > > > > debug a
> > > > > > Stack
> > > > > > Overflow in JRockit, but the ones I saw on the sun jdk do show the
> > > > > > stack
> > > > > > trace, and then its easy to understand where it is coming from, as
> > > > > > you
> > > > > > see
> > > > > > the method(s) that keep repeating.
>
> > > > > > The [tp] thread pool is the main thread pool in elasticsearch.
>
> > > > > > -shay.banon
>
> > > > > > On Sat, Nov 6, 2010 at 11:00 PM, Sebastian [sgavar...@gmail.com](mailto:sgavar...@gmail.com)
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Shay,
>
> ...
>
> read more »
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:16am UTC](https://discuss.elastic.co/t/stackoverflowerror-after-node-connect-java-api/3529/11 "2017-07-06T04:16:58Z")