# Elasticsearch process exits abnormally

**URL:** <https://discuss.elastic.co/t/elasticsearch-process-exits-abnormally/267603>\
**Category:** Elasticsearch\
**Created:** [March 18, 2021, 6:55am UTC](https://discuss.elastic.co/t/elasticsearch-process-exits-abnormally/267603 "2021-03-18T06:55:32Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![bxl](https://avatars.discourse-cdn.com/v4/letter/b/aca169/32.png) [@bxl](https://discuss.elastic.co/u/bxl)\
**Post date:** [March 18, 2021, 6:55am UTC](https://discuss.elastic.co/t/elasticsearch-process-exits-abnormally/267603/1 "2021-03-18T06:55:32Z")

</div>

elasticsearch process exits abnormally  
system：CentOS Linux release 7.4.1708 (Core)  
3.10.0-693.el7.x86\_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86\_64 x86\_64 x86\_64 GNU/Linux  
openjdk version "1.8.0\_252"  
OpenJDK Runtime Environment (build 1.8.0\_252-b09)  
OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)

Mar 18 14:41:51 GL-BPL1001494 systemd: elasticsearch.service: main process exited, code=killed, status=6/ABRT  
Mar 18 14:41:51 GL-BPL1001494 systemd: Unit elasticsearch.service entered failed state.  
Mar 18 14:41:51 GL-BPL1001494 systemd: elasticsearch.service failed.  
hs\_err\_pid120106.log

```auto
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007fea10b954dc, pid=120106, tid=0x00007fe8da735700
#
# JRE version: OpenJDK Runtime Environment (8.0_252-b09) (build 1.8.0_252-b09)
# Java VM: OpenJDK 64-Bit Server VM (25.252-b09 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C [libc.so.6+0x804dc] cfree+0x1c
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
#

--------------- T H R E A D ---------------

Current thread (0x00007fea0811e000): ConcurrentGCThread [stack: 0x00007fe8da635000,0x00007fe8da736000] [id=120418]

siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x00007fe8d8066ff8

Registers:
RAX=0x0000000000000000, RBX=0x00007fdff8295630, RCX=0x0000000000000001, RDX=0x00007fe8d8067000
RSP=0x00007fe8da734c48, RBP=0x00000000ffffffff, RSI=0x00007fe8d8067000, RDI=0x00007fe8d8067000
R8 =0xffffffffffffffff, R9 =0x0000000000000000, R10=0x0000000000000022, R11=0x0000000000000246
R12=0x0000000000000000, R13=0x000000000000001c, R14=0x0000000000000000, R15=0x00007fea0811e000
RIP=0x00007fea10b954dc, EFLAGS=0x0000000000010206, CSGSFS=0x0000000000000033, ERR=0x0000000000000004
  TRAPNO=0x000000000000000e

Top of Stack: (sp=0x00007fe8da734c48)
0x00007fe8da734c48: 00007fea10b8d13a 00007fdff8295630
0x00007fe8da734c58: 00007fea10b8c8d5 000000000000001c
0x00007fe8da734c68: 00007fdff8295630 00007fe8da734d20
0x00007fe8da734c78: 00007fea10b8b651 000000000000002b
0x00007fe8da734c88: 00007fdff8295630 0000000000000001
0x00007fe8da734c98: 000000000000001c 000000000000001c
0x00007fe8da734ca8: 00007fea0805c1e0 00007fea0811e000
0x00007fe8da734cb8: 00007fea10b8103d 0000007900000002
0x00007fe8da734cc8: 00007fea080051f0 00007fe8da734d10
0x00007fe8da734cd8: 000000000000001c 00007fe8da734d20
0x00007fe8da734ce8: 00007fea103deeac 0000000000000000
0x00007fe8da734cf8: 00007fea080051f0 00007fea080b2278
0x00007fe8da734d08: 00007fe8da734d20 00007fe8da734d70
0x00007fe8da734d18: 00007fea103e0959 2d33302d31323032
0x00007fe8da734d28: 31343a3431543831 2b3937322e31343a
0x00007fe8da734d38: 00007f0030303830 3fc999999999999a
0x00007fe8da734d48: 93ed24f45810b900 00000000024d9263
0x00007fe8da734d58: 00007fea10aec6e0 00007fea10af66e0
0x00007fe8da734d68: 00007fea0811e7f0 00007fe8da734ed0
0x00007fe8da734d78: 00007fea0ffef985 00007fea08061990
0x00007fe8da734d88: 00000000000003d8 00007fea0811e000
0x00007fe8da734d98: 00007fea0811e7f0 00007fe8da734e10
0x00007fe8da734da8: 00007fea0811ec18 00007fea0811e840
0x00007fe8da734db8: 00007fea08062160 00007fea0811e830
0x00007fe8da734dc8: 413e99a5867c50d2 3fe0000000000000
0x00007fe8da734dd8: 404e42f68fdc65c4 3fe0000000000000
0x00007fe8da734de8: 413e99ab03913c17 40d227adaf5ba2be
0x00007fe8da734df8: 413e99ab01fd0e1e 00007fea10a9eb20
0x00007fe8da734e08: 00007fea0811d400 00007fea0811e000
0x00007fe8da734e18: 00007fea0811ec20 00007fea0811ec60
0x00007fe8da734e28: 00007fea0811ec70 00007fea0811ed48
0x00007fe8da734e38: 00000000000000d8 00007fea0811ee80 

Instructions: (pc=0x00007fea10b954dc)
0x00007fea10b954bc: 1f 44 00 00 48 8b 05 21 ba 33 00 48 8b 00 48 85
0x00007fea10b954cc: c0 0f 85 bf 00 00 00 48 85 ff 0f 84 b4 00 00 00
0x00007fea10b954dc: 48 8b 47 f8 48 8d 4f f0 a8 02 75 28 a8 04 48 8d
0x00007fea10b954ec: 3d 6f c2 33 00 74 0c 48 89 c8 48 25 00 00 00 fc 

Register to memory mapping:

RAX=0x0000000000000000 is an unknown value
RBX=0x00007fdff8295630 is an unknown value
RCX=0x0000000000000001 is an unknown value
RDX=0x00007fe8d8067000 is an unknown value
RSP=0x00007fe8da734c48 is an unknown value
RBP=0x00000000ffffffff is an unknown value
RSI=0x00007fe8d8067000 is an unknown value
RDI=0x00007fe8d8067000 is an unknown value
R8 =0xffffffffffffffff is an unknown value
R9 =0x0000000000000000 is an unknown value
R10=0x0000000000000022 is an unknown value
R11=0x0000000000000246 is an unknown value
R12=0x0000000000000000 is an unknown value
R13=0x000000000000001c is an unknown value
R14=0x0000000000000000 is an unknown value
R15=0x00007fea0811e000 is an unknown value

Stack: [0x00007fe8da635000,0x00007fe8da736000], sp=0x00007fe8da734c48, free space=1023k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [libc.so.6+0x804dc] cfree+0x1c

```

---

<div class="post-metadata">

**Author:** ![bxl](https://avatars.discourse-cdn.com/v4/letter/b/aca169/32.png) [@bxl](https://discuss.elastic.co/u/bxl)\
**Post date:** [March 18, 2021, 7:05am UTC](https://discuss.elastic.co/t/elasticsearch-process-exits-abnormally/267603/2 "2021-03-18T07:05:48Z")

</div>

elasticsearch log

```auto
[2021-03-18T14:28:47,225][INFO][o.e.m.j.JvmGcMonitorService] [10.0.1.1_node] [gc][2001256] overhead, spent [629ms] collecting in the last [1.2s]
[2021-03-18T14:32:10,880][INFO][o.e.m.j.JvmGcMonitorService] [10.0.1.1_node] [gc][2001459] overhead, spent [256ms] collecting in the last [1s]
[2021-03-18T14:32:40,900][INFO][o.e.m.j.JvmGcMonitorService] [10.0.1.1_node] [gc][2001489] overhead, spent [376ms] collecting in the last [1s]
[2021-03-18T14:33:11,906][INFO][o.e.m.j.JvmGcMonitorService] [10.0.1.1_node] [gc][2001520] overhead, spent [297ms] collecting in the last [1s]
[2021-03-18T14:33:46,042][INFO][o.e.m.j.JvmGcMonitorService] [10.0.1.1_node] [gc][2001554] overhead, spent [270ms] collecting in the last [1s]
[2021-03-18T14:34:54,170][INFO][o.e.m.j.JvmGcMonitorService] [10.0.1.1_node] [gc][2001622] overhead, spent [279ms] collecting in the last [1s]
[2021-03-18T14:36:59,812][INFO][o.e.m.j.JvmGcMonitorService] [10.0.1.1_node] [gc][2001747] overhead, spent [341ms] collecting in the last [1s]
[2021-03-18T14:39:10,513][INFO][o.e.m.j.JvmGcMonitorService] [10.0.1.1_node] [gc][2001877] overhead, spent [266ms] collecting in the last [1s]
[2021-03-18T14:41:25,000][INFO][o.e.m.j.JvmGcMonitorService] [10.0.1.1_node] [gc][2002011] overhead, spent [284ms] collecting in the last [1s]
[2021-03-18T14:42:27,297][INFO][o.e.e.NodeEnvironment] [10.0.1.1_node] using [2] data paths, mounts [[/ssd1 (/dev/nvme0n1), /ssd2 (/dev/nvme1n1)]], net usable_space [4.1tb], net total_space [5.8tb], types [xfs]
[2021-03-18T14:42:27,300][INFO][o.e.e.NodeEnvironment] [10.0.1.1_node] heap size [31gb], compressed ordinary object pointers [true]
[2021-03-18T14:42:27,362][INFO][o.e.n.Node] [10.0.1.1_node] node name [10.0.1.1_node], node ID [E3nPLjQ7TEKkjI4ZroMTzQ]
[2021-03-18T14:42:27,362][INFO][o.e.n.Node] [10.0.1.1_node] version[6.8.1], pid[63458], build[default/rpm/1fad4e1/2019-06-18T13:16:52.517138Z], OS[Linux/3.10.0-693.el7.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_252/25.252-b09]
[2021-03-18T14:42:27,362][INFO][o.e.n.Node] [10.0.1.1_node] JVM arguments [-Xms31g, -Xmx31g, -XX:+UseG1GC, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-5262983360642704363, -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:/var/log/elasticsearch/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=default, -Des.distribution.type=rpm]
[2021-03-18T14:42:30,564][INFO][o.e.p.PluginsService] [10.0.1.1_node] loaded module [aggs-matrix-stats]
[2021-03-18T14:42:30,564][INFO][o.e.p.PluginsService] [10.0.1.1_node] loaded module [analysis-common]
[2021-03-18T14:42:30,564][INFO][o.e.p.PluginsService] [10.0.1.1_node] loaded module [ingest-common]
[2021-03-18T14:42:30,564][INFO][o.e.p.PluginsService] [10.0.1.1_node] loaded module [ingest-geoip]
[2021-03-18T14:42:30,564][INFO][o.e.p.PluginsService] [10.0.1.1_node] loaded module [ingest-user-agent]
[2021-03-18T14:42:30,564][INFO][o.e.p.PluginsService] [10.0.1.1_node] loaded module [lang-expression]

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 15, 2021, 7:06am UTC](https://discuss.elastic.co/t/elasticsearch-process-exits-abnormally/267603/3 "2021-04-15T07:06:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
