ES-5.2.1 server is continuously getting dead

Elastic Stack Elasticsearch
1

LOGS
[2017-02-20T08:55:20,322][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [abc] fatal error in thread [elasticsearch[abc][search][T#7]], exiting
java.lang.StackOverflowError: null
at org.apache.lucene.util.automaton.Operations.topoSortStatesRecurse(Operations.java:1309) ~[lucene-core-6.4.1.jar:6.4.1 72f75b2503fa0aa4f0aff76d439874feb923bb0e - jpountz - 2017-02-01 14:43:32]
at org.apache.lucene.util.automaton.Operations.topoSortStatesRecurse(Operations.java:1309) ~[lucene-core-6.4.1.jar:6.4.1 72f75b2503fa0aa4f0aff76d439874feb923bb0e - jpountz - 2017-02-01 14:43:32]
at org.apache.lucene.util.automaton.Operations.topoSortStatesRecurse(Operations.java:1309) ~[lucene-core-6.4.1.jar:6.4.1 72f75b2503fa0aa4f0aff76d439874feb923bb0e - jpountz - 2017-02-01 14:43:32]

Could you please let me know, What exaclty is the isssue here.

2

What OS? What JVM?

3

I may confirm the issue for version[5.2.0]:
[2017-02-25T12:22:12,274][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [Node2] fatal error in thread [elasticsearch[Node2][search][T#11]], exiting
java.lang.StackOverflowError: null
at org.apache.lucene.util.automaton.Operations.topoSortStatesRecurse(Operations.java:1309) ~[lucene-core-6.4.0.jar:6.4.0 bbe4b08cc1fb673d0c3eb4b8455f23ddc1364124 - jim - 2017-01-17 15:57:29]
at org.apache.lucene.util.automaton.Operations.topoSortStatesRecurse(Operations.java:1309) ~[lucene-core-6.4.0.jar:6.4.0 bbe4b08cc1fb673d0c3eb4b8455f23ddc1364124 - jim - 2017-01-17 15:57:29]
at org.apache.lucene.util.automaton.Operations.topoSortStatesRecurse(Operations.java:1309) ~[lucene-core-6.4.0.jar:6.4.0 bbe4b08cc1fb673d0c3eb4b8455f23ddc1364124 - jim - 2017-01-17 15:57:29]
at org.apache.lucene.util.automaton.Operations.topoSortStatesRecurse(Operations.java:1309) ~[lucene-core-6.4.0.jar:6.4.0 bbe4b08cc1fb673d0c3eb4b8455f23ddc1364124 - jim - 2017-01-17 15:57:29]
at org.apache.lucene.util.automaton.Operations.topoSortStatesRecurse(Operations.java:1309) ~[lucene-core-6.4.0.jar:6.4.0 bbe4b08cc1fb673d0c3eb4b8455f23ddc1364124 - jim - 2017-01-17 15:57:29]

root@n1 ~ # uname -v
#1 SMP Debian 3.16.7-ckt25-2+deb8u3 (2016-07-02)

root@n1 ~ # lsb_release -da
Distributor ID: Debian
Description: Debian GNU/Linux 8.5 (jessie)
Release: 8.5
Codename: jessie

root@n1 / # java -version
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (build 1.8.0_121-8u121-b13-1~bpo8+1-b13)
OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)

4

Can you share the entire log, things before and after this as well.

5

Log file
Here is the log file.
As you may see, lines before the SOE seem ordinary, nothing special happens.
The last line is 1.5 hours later, when I started the node.

6

More info about ES installation:
"version": {
"number": "5.2.0",
"build_hash": "24e05b9",
"build_date": "2017-01-24T19:52:35.800Z",
"build_snapshot": false,
"lucene_version": "6.4.0"
},

7

Same issue occured today with the same set of logs

Same issue is also faced by solr guys http://lucene.472066.n3.nabble.com/StackOverflowError-when-building-suggestions-td4267700.html

8

I'm having the same problem, for every few days my service is getting dead.

1 Like
9

@Akhilesh_Anb Please share the cluster logs and settings

10

java -version
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (build 1.8.0_121-b13)
OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)

NAME="Amazon Linux AMI"
VERSION="2016.09"
ID="amzn"
ID_LIKE="rhel fedora"
VERSION_ID="2016.09"
Amazon Linux AMI release 2016.09

11

Here is the link of log file

java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)

Os :- Centos 64 bit
Elasticsearch - 5.1.2

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.