# Elasticsearch crashes with StackOverflowError
**URL:** https://discuss.elastic.co/t/elasticsearch-crashes-with-stackoverflowerror/102896
**Category:** Elasticsearch
**Created:** [October 5, 2017, 6:59pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-stackoverflowerror/102896 "2017-10-05T18:59:27Z")
**Posts on this page:** 14
**Page:** 1
### Author:  [@praveenmak](https://discuss.elastic.co/u/praveenmak)
#### Post date: [October 5, 2017, 6:59pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-stackoverflowerror/102896/1 "2017-10-05T18:59:27Z")
Version Used: ElasticSearch 5.6.6
Mode: Single Node, Single Cluster - config is simple as shown below.
**node.master: true**
**node.data: true**
**transport.type: local # disable network**
**bootstrap.memory\_lock: true**
**thread\_pool.search.queue\_size: 10000**
I am trying to search in Kibana with "esplogs-\*", there are 75 Million records.
Here is the stacktrace after searching,
[2017-10-05T14:50:07,327][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [localhost] fatal error in thread [elasticsearch[localhost][search][T#19]], exiting
java.lang.StackOverflowError: null
at java.util.Collections$UnmodifiableCollection.size(Collections.java:1030) ~[?:1.8.0\_144]
at java.util.Collections$UnmodifiableCollection.size(Collections.java:1030) ~[?:1.8.0\_144]
at org.elasticsearch.cluster.routing.PlainShardsIterator.remaining(PlainShardsIterator.java:50) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:196) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.2.jar:5.6.2]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.2.jar:5.6.2]
Hoping if someone helps. Please let me know what you need from me.
---
### Author:  [@praveenmak](https://discuss.elastic.co/u/praveenmak)
#### Post date: [October 5, 2017, 10:51pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-stackoverflowerror/102896/2 "2017-10-05T22:51:46Z")
How do I check how many shards are occupied..
I keep getting this error whenever I insert loads of data. I have created 24 indexes and I have 5 shards (default).
Is there a better configuration to handle single node with large data?
Please let me know.
---
### Author:  [@Rohithnama](https://discuss.elastic.co/u/Rohithnama)
#### Post date: [October 5, 2017, 11:06pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-stackoverflowerror/102896/3 "2017-10-05T23:06:18Z")
> [@praveenmak](#):
>
> I keep getting this error whenever I insert loads of data. I have created 24 indexes and I have 5 shards (default).
>
> Is there a better configuration to handle single node with large data?
I had the same issue but I have an option to delete the indexes anytime so whenever I encountered this scenario, I deleted the indices- it is a temporary solution. If you don't mind deleting the some of the indexes just give a shot!
The path to delete is ~/elasticsearch-5.6.2/data/nodes/0/indices. I'm not sure if this could resolve your issues but it's worth the try!
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [October 5, 2017, 11:25pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-stackoverflowerror/102896/4 "2017-10-05T23:25:16Z")
> [@praveenmak](#):
>
> Elasticsearch 5.6.6
Please check again, we are only at 5.6.2.
> [@praveenmak](#):
>
> thread\_pool.search.queue\_size: 10000
That is dangerous, why did you increase it?
Is there more errors at this time? What do the Kibana logs show?
> [@Rohithnama](#):
>
> The path to delete is ~/elasticsearch-5.6.2/data/nodes/0/indices. I'm not sure if this could resolve your issues but it's worth the try!
Don't do that, it's very dangerous and can cause data loss. If you need to delete data, use the APIs.
---
### Author:  [@praveenmak](https://discuss.elastic.co/u/praveenmak)
#### Post date: [October 6, 2017, 3:34pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-stackoverflowerror/102896/5 "2017-10-06T15:34:32Z")
Thanks @Rohithnama and @warkolm for response.
You are right my bad, I typed it wrong its 5.6.2
**/usr/share/elasticsearch/bin/elasticsearch --version**
**Version: 5.6.2, Build: 57e20f3/2017-09-23T13:16:45.703Z, JVM: 1.8.0\_144**
I saw a suggestion , increasing the thread\_pool.search.queue\_size fixed the slowness.
**[https://stackoverflow.com/questions/30053967/courier-fetch-shards-failed](https://stackoverflow.com/questions/30053967/courier-fetch-shards-failed)**
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [October 6, 2017, 8:36pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-stackoverflowerror/102896/6 "2017-10-06T20:36:14Z")
No, it just overloads things even more.
---
### Author:  [@praveenmak](https://discuss.elastic.co/u/praveenmak)
#### Post date: [October 13, 2017, 3:03pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-stackoverflowerror/102896/7 "2017-10-13T15:03:33Z")
@warkolm
I am seeing another crash , I had reduced the queue size to 1000. Could you please help.
[2017-10-13T10:46:38,970][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [bctlppostalclean18] fatal error in thread [elasticsearch[bctlppostalclean18][search][T#10]], exiting
java.lang.StackOverflowError: null
at org.elasticsearch.cluster.routing.PlainShardsIterator.remaining(PlainShardsIterator.java:50) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:196) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.AbstractSearchAsyncAction.skipShard(AbstractSearchAsyncAction.java:321) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.maybeExecuteNext(InitialSearchPhase.java:147) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.successfulShardExecution(InitialSearchPhase.java:207) ~[elasticsearch-5.6.3.jar:5.6.3]
at org.elasticsearch.action.search.InitialSearchPhase.skipShard(InitialSearchPhase.java:323) ~[elasticsearch-5.6.3.jar:5.6.3]
---
### Author:  [@praveenmak](https://discuss.elastic.co/u/praveenmak)
#### Post date: [October 17, 2017, 7:08pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-stackoverflowerror/102896/8 "2017-10-17T19:08:07Z")
Please help , is this a known issue?
---
### Author:  [@erley](https://discuss.elastic.co/u/erley)
#### Post date: [October 24, 2017, 3:36pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-stackoverflowerror/102896/9 "2017-10-24T15:36:40Z")
We have the same problem (ES v5.6.2).
A simple license install breaks ES nodes with StackOverflowError ☹
Currently we had to rollback to v5.5.2 - the last known to work correctly...
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [October 24, 2017, 6:15pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-stackoverflowerror/102896/10 "2017-10-24T18:15:42Z")
Please create a new thread with some more detail, it may not be related to this one so separating it out would be super helpful in trying to find the problem 🙂
---
### Author:  [@praveenmak](https://discuss.elastic.co/u/praveenmak)
#### Post date: [October 24, 2017, 8:01pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-stackoverflowerror/102896/11 "2017-10-24T20:01:51Z")
@warkolm
May I know if this issue is a known issue or not?
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [October 24, 2017, 8:26pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-stackoverflowerror/102896/12 "2017-10-24T20:26:57Z")
I haven't seen it before, but that's not a definitive answer sorry. Did you look on github?
---
### Author:  [@praveenmak](https://discuss.elastic.co/u/praveenmak)
#### Post date: [October 24, 2017, 8:30pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-stackoverflowerror/102896/13 "2017-10-24T20:30:14Z")
Could not find this in Github ☹
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 21, 2017, 8:30pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-stackoverflowerror/102896/14 "2017-11-21T20:30:58Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.