# Elasticsearch overhead all the time

**URL:** https://discuss.elastic.co/t/elasticsearch-overhead-all-the-time/156571
**Category:** Elasticsearch
**Created:** [November 14, 2018, 2:56am UTC](https://discuss.elastic.co/t/elasticsearch-overhead-all-the-time/156571 "2018-11-14T02:56:25Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![liuyang1943](https://avatars.discourse-cdn.com/v4/letter/l/aeb1de/32.png) [@liuyang1943](https://discuss.elastic.co/u/liuyang1943)
#### Post date: [November 14, 2018, 2:56am UTC](https://discuss.elastic.co/t/elasticsearch-overhead-all-the-time/156571/1 "2018-11-14T02:56:25Z")

</div>

I deployed ELK in one node, and configured the Elasticsearch's memory with 10G, no other changes for ES.  
But every time, when I perform the query from Kibana, the ES always reported the error as 🙂  
[2018-11-14T10:38:42,660][INFO][o.e.g.GatewayService] [ug\_uKSM] recovered [2] indices into cluster\_state  
[2018-11-14T10:38:43,489][INFO][o.e.c.r.a.AllocationService] [ug\_uKSM] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[logmonster][2]] ...]).  
[2018-11-14T10:39:34,222][INFO][o.e.m.j.JvmGcMonitorService] [ug\_uKSM] [gc][55] overhead, spent [672ms] collecting in the last [1.5s]  
[2018-11-14T10:39:35,469][WARN][o.e.m.j.JvmGcMonitorService] [ug\_uKSM] [gc][56] overhead, spent [817ms] collecting in the last [1.2s]  
[2018-11-14T10:39:36,631][WARN][o.e.m.j.JvmGcMonitorService] [ug\_uKSM] [gc][57] overhead, spent [808ms] collecting in the last [1.1s]  
[2018-11-14T10:39:37,751][WARN][o.e.m.j.JvmGcMonitorService] [ug\_uKSM] [gc][58] overhead, spent [822ms] collecting in the last [1.1s]  
[2018-11-14T10:39:38,782][WARN][o.e.m.j.JvmGcMonitorService] [ug\_uKSM] [gc][59] overhead, spent [705ms] collecting in the last [1s]  
[2018-11-14T10:39:39,783][WARN][o.e.m.j.JvmGcMonitorService] [ug\_uKSM] [gc][60] overhead, spent [618ms] collecting in the last [1s]  
[2018-11-14T10:39:40,784][WARN][o.e.m.j.JvmGcMonitorService] [ug\_uKSM] [gc][61] overhead, spent [716ms] collecting in the last [1s]  
[2018-11-14T10:39:41,785][WARN][o.e.m.j.JvmGcMonitorService] [ug\_uKSM] [gc][62] overhead, spent [504ms] collecting in the last [1s]  
[2018-11-14T10:39:42,786][INFO][o.e.m.j.JvmGcMonitorService] [ug\_uKSM] [gc][63] overhead, spent [452ms] collecting in the last [1s]

anything else I missed? any one can give some comments for this problem?  
Thanks in advance.

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [November 14, 2018, 5:49am UTC](https://discuss.elastic.co/t/elasticsearch-overhead-all-the-time/156571/2 "2018-11-14T05:49:38Z")

</div>

What version are you on?  
How many shards, how many indices, how many GB of data?

---

<div class="post-metadata">

### Author: ![liuyang1943](https://avatars.discourse-cdn.com/v4/letter/l/aeb1de/32.png) [@liuyang1943](https://discuss.elastic.co/u/liuyang1943)
#### Post date: [November 14, 2018, 6:06am UTC](https://discuss.elastic.co/t/elasticsearch-overhead-all-the-time/156571/3 "2018-11-14T06:06:30Z")

</div>

The ES version is 6.4.1, and just one index, and one node, around 100G memory in my node, but in ES's configuration, allocated 10G for ES:  
[root@localhost logs]# cat /home/11thone/elk/elasticsearch/config/jvm.options  
## JVM configuration

```
################################################################
## IMPORTANT: JVM heap size
################################################################
##
## You should always set the min and max JVM heap
## size to the same value. For example, to set
## the heap to 4 GB, set:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
## for more information
##
################################################################

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms10g
-Xmx10g

################################################################
## Expert settings
################################################################
##
## All settings below this section are considered
## expert settings. Don't tamper with them unless
## you understand what you are doing
##
################################################################

## GC configuration
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly

## optimizations
# pre-touch memory pages used by the JVM during initialization
-XX:+AlwaysPreTouch

## basic

# explicitly set the stack size
-Xss5m

# set to headless, just in case
-Djava.awt.headless=true

# ensure UTF-8 encoding by default (e.g. filenames)
-Dfile.encoding=UTF-8

# use our provided JNA always versus the system one
-Djna.nosys=true

# turn off a JDK optimization that throws away stack traces for common
# exceptions because stack traces are important for debugging
-XX:-OmitStackTraceInFastThrow

# flags to configure Netty
-Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true
-Dio.netty.recycler.maxCapacityPerThread=0

# log4j 2
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true

-Djava.io.tmpdir=${ES_TMPDIR}

## heap dumps

# generate a heap dump when an allocation from the Java heap fails
# heap dumps are created in the working directory of the JVM
-XX:+HeapDumpOnOutOfMemoryError

# specify an alternative path for heap dumps; ensure the directory exists and
# has sufficient space
-XX:HeapDumpPath=data

# specify an alternative path for JVM fatal error logs
-XX:ErrorFile=logs/hs_err_pid%p.log

## JDK 8 GC logging

8:-XX:+PrintGCDetails
8:-XX:+PrintGCDateStamps
8:-XX:+PrintTenuringDistribution
8:-XX:+PrintGCApplicationStoppedTime
8:-Xloggc:logs/gc.log
8:-XX:+UseGCLogFileRotation
8:-XX:NumberOfGCLogFiles=32
8:-XX:GCLogFileSize=64m

# JDK 9+ GC logging
9-:-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m
# due to internationalization enhancements in JDK 9 Elasticsearch need to set the provider to COMPAT 
otherwise
# time/date parsing will break in an incompatible way for some date patterns and locals
9-:-Djava.locale.providers=COMPAT

# temporary workaround for C2 bug with JDK 10 on hardware with AVX-512
10-:-XX:UseAVX=2
```

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [November 14, 2018, 8:02am UTC](https://discuss.elastic.co/t/elasticsearch-overhead-all-the-time/156571/4 "2018-11-14T08:02:45Z")

</div>

Can you please use the format button - `</>` - on that code, it's very hard to read.

---

<div class="post-metadata">

### Author: ![liuyang1943](https://avatars.discourse-cdn.com/v4/letter/l/aeb1de/32.png) [@liuyang1943](https://discuss.elastic.co/u/liuyang1943)
#### Post date: [November 14, 2018, 9:41am UTC](https://discuss.elastic.co/t/elasticsearch-overhead-all-the-time/156571/5 "2018-11-14T09:41:33Z")

</div>

Hi, warkolm

Thanks, I re-edited it, please help check it.

---

<div class="post-metadata">

### Author: ![liuyang1943](https://avatars.discourse-cdn.com/v4/letter/l/aeb1de/32.png) [@liuyang1943](https://discuss.elastic.co/u/liuyang1943)
#### Post date: [November 16, 2018, 7:24am UTC](https://discuss.elastic.co/t/elasticsearch-overhead-all-the-time/156571/6 "2018-11-16T07:24:48Z")

</div>

Hi, Warkolm

Finally, I still cannot found out the reason of this problem, hence I have to re-install a new version of ELK stack. after that, it's back to work.

Anyway, thanks for your focus and help, have a nice day.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 14, 2018, 7:34am UTC](https://discuss.elastic.co/t/elasticsearch-overhead-all-the-time/156571/7 "2018-12-14T07:34:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
