# Elastic search fails after some time

**URL:** https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965
**Category:** Elasticsearch
**Created:** [January 24, 2016, 6:10am UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965 "2016-01-24T06:10:07Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![ambarishrh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ambarishrh/32/7348_2.png) [@ambarishrh](https://discuss.elastic.co/u/ambarishrh)
#### Post date: [January 24, 2016, 6:10am UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/1 "2016-01-24T06:10:07Z")

</div>

Hello,

We have elasticsearch installed and configured to work with ActiveCollab on CentOS7 64 bit with 8GB RAM. ElasticSearch starts successfully but fails after some time. The status message shows:

elasticsearch.service - Elasticsearch  
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)  
Active: failed (Result: signal) since Thu 2016-01-21 11:05:55 GST; 1h 17min ago  
Docs: [http://www.elastic.co](http://www.elastic.co)  
Main PID: 206246 (code=killed, signal=ABRT)  
Started Elasticsearch.  
Starting Elasticsearch...  
elasticsearch.service: main process exited, code=killed, status=6/ABRT  
Unit elasticsearch.service entered failed state.  
elasticsearch.service failed.

Couldn't find the reason/fix yet. Please help

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [January 24, 2016, 6:47am UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/2 "2016-01-24T06:47:16Z")

</div>

> [@ambarishrh](#):
>
> elasticsearch.service: main process exited, code=killed, status=6/ABRT

Something is killing it by the looks of things.  
Try checking your primary system logs.

---

<div class="post-metadata">

### Author: ![ambarishrh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ambarishrh/32/7348_2.png) [@ambarishrh](https://discuss.elastic.co/u/ambarishrh)
#### Post date: [January 24, 2016, 7:12am UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/3 "2016-01-24T07:12:26Z")

</div>

Didn't find anything specific from the logs. Is there any specific things i can check to find this?

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [January 24, 2016, 7:12am UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/4 "2016-01-24T07:12:58Z")

</div>

Which logs did you check?

---

<div class="post-metadata">

### Author: ![ambarishrh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ambarishrh/32/7348_2.png) [@ambarishrh](https://discuss.elastic.co/u/ambarishrh)
#### Post date: [January 24, 2016, 7:14am UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/5 "2016-01-24T07:14:28Z")

</div>

/var/log/elasticsearch/elasticsearch

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [January 24, 2016, 7:15am UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/6 "2016-01-24T07:15:17Z")

</div>

And what about the system level logs, as I mentioned.

---

<div class="post-metadata">

### Author: ![ambarishrh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ambarishrh/32/7348_2.png) [@ambarishrh](https://discuss.elastic.co/u/ambarishrh)
#### Post date: [January 24, 2016, 7:16am UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/7 "2016-01-24T07:16:44Z")

</div>

Checked messages, if thats what you are asking about, but that gives the same as elasticsearch status

systemd: elasticsearch.service: main process exited, code=killed, status=6/ABRT  
systemd: Unit elasticsearch.service entered failed state.  
systemd: elasticsearch.service failed.

---

<div class="post-metadata">

### Author: ![ambarishrh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ambarishrh/32/7348_2.png) [@ambarishrh](https://discuss.elastic.co/u/ambarishrh)
#### Post date: [January 24, 2016, 7:49am UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/8 "2016-01-24T07:49:54Z")

</div>

Just started journalctl -u elasticsearch.service -f, will see if it shows anything more when this fails. As per one post, i just changed the StandardOutput=null to StandardOutput=journal

---

<div class="post-metadata">

### Author: ![ambarishrh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ambarishrh/32/7348_2.png) [@ambarishrh](https://discuss.elastic.co/u/ambarishrh)
#### Post date: [January 26, 2016, 6:25pm UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/9 "2016-01-26T18:25:40Z")

</div>

I got some detailed error which relates to Java.

# 

# A fatal error has been detected by the Java Runtime Environment:

# 

# SIGSEGV (0xb) at pc=0x00007f3d5ec2a3cc, pid=971889, tid=139901534000896

# 

# JRE version: Java(TM) SE Runtime Environment (8.0\_60-b27) (build 1.8.0\_60-b27)

# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.60-b23 mixed mode linux-amd64 compressed oops)

# Problematic frame:

# V [libjvm.so+0x93c3cc] ParRootScanWithoutBarrierClosure::do\_oop(oopDesc\*\*)+0x6c

# 

# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again

# 

# If you would like to submit a bug report, please visit:

# [http://bugreport.java.com/bugreport/crash.jsp](http://bugreport.java.com/bugreport/crash.jsp)

# 

--------------- T H R E A D ---------------

Current thread (0x00007f3d5801a000): GCTaskThread [stack: 0x00007f3d5d2cd000,0x00007f3d5d3ce000] [id=971897]

siginfo: si\_signo: 11 (SIGSEGV), si\_code: 1 (SEGV\_MAPERR), si\_addr: 0x0000000002400288

---

<div class="post-metadata">

### Author: ![ambarishrh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ambarishrh/32/7348_2.png) [@ambarishrh](https://discuss.elastic.co/u/ambarishrh)
#### Post date: [January 28, 2016, 5:19pm UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/10 "2016-01-28T17:19:26Z")

</div>

Is there anything else that i could check? Increased the heap size to half of the server RAM but it still failed!

---

<div class="post-metadata">

### Author: ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)
#### Post date: [January 28, 2016, 6:43pm UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/11 "2016-01-28T18:43:50Z")

</div>

Oh dear, looks like you found a bug in the JVM. 😫

I skimmed the jvm bugtracker and didn't see a related bug, but I may have missed it. The easiest route is to upgrade your JVM to most recent (1\_8\_u71) and hope the bug is fixed.

Are you using G1GC by chance? The segfault was thrown from a GC thread, and it's known that G1GC is not stable yet and throws segfaults on occasion 🙂

---

<div class="post-metadata">

### Author: ![ambarishrh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ambarishrh/32/7348_2.png) [@ambarishrh](https://discuss.elastic.co/u/ambarishrh)
#### Post date: [January 28, 2016, 8:37pm UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/12 "2016-01-28T20:37:22Z")

</div>

Thanks for the help. I am not very sure on how to upgrade to the recent java version. Followed the documentation from [https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-elasticsearch-on-centos-7](https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-elasticsearch-on-centos-7)

---

<div class="post-metadata">

### Author: ![ambarishrh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ambarishrh/32/7348_2.png) [@ambarishrh](https://discuss.elastic.co/u/ambarishrh)
#### Post date: [January 31, 2016, 6:15pm UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/13 "2016-01-31T18:15:42Z")

</div>

I installed ES on a separate server to avoid resources being shared with anything else. Now i need to give access to a remote server to ES, I couldn't figure out how to do that. Ports are opened but can't connect. Please help

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [January 31, 2016, 6:29pm UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/14 "2016-01-31T18:29:59Z")

</div>

Did you change network.host to bind to anything but localhost?

---

<div class="post-metadata">

### Author: ![ambarishrh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ambarishrh/32/7348_2.png) [@ambarishrh](https://discuss.elastic.co/u/ambarishrh)
#### Post date: [January 31, 2016, 6:39pm UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/15 "2016-01-31T18:39:17Z")

</div>

Below are my settings now:

network.bind\_host: localhost  
network.publish\_host: localhost  
network.host: localhost

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [January 31, 2016, 6:52pm UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/16 "2016-01-31T18:52:12Z")

</div>

Change network.host and remove the 2 others.

---

<div class="post-metadata">

### Author: ![ambarishrh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ambarishrh/32/7348_2.png) [@ambarishrh](https://discuss.elastic.co/u/ambarishrh)
#### Post date: [January 31, 2016, 6:52pm UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/17 "2016-01-31T18:52:50Z")

</div>

Thanks, change the network.host to the Public IP of the server?

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [January 31, 2016, 6:53pm UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/18 "2016-01-31T18:53:33Z")

</div>

Public or private. But to an IP your other server can see.

---

<div class="post-metadata">

### Author: ![ambarishrh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ambarishrh/32/7348_2.png) [@ambarishrh](https://discuss.elastic.co/u/ambarishrh)
#### Post date: [January 31, 2016, 6:59pm UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/19 "2016-01-31T18:59:46Z")

</div>

Thank you so much, that worked! 🙂

---

<div class="post-metadata">

### Author: ![ambarishrh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ambarishrh/32/7348_2.png) [@ambarishrh](https://discuss.elastic.co/u/ambarishrh)
#### Post date: [January 31, 2016, 7:25pm UTC](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965/20 "2016-01-31T19:25:50Z")

</div>

I am able to access the elastic search on public IP on port 9200, but unable to connect ActiveCollab to ES. Still checking this now

[Next page](https://discuss.elastic.co/t/elastic-search-fails-after-some-time/39965.md?page=2)
