Elasticsearch overcommit (VSZ) 300-900%

Elastic Stack Elasticsearch
1

Hello.

I dont know why i have very high VSZ (virtual memory size) ~ 800-900%, heap 4gb in settings

 PID  PPID CMD                                           RSS    VSZ %MEM %CPU
11554     1 /usr/bin/java -Xms4g -Xmx4g java            4974936 46637680 36.3 0.0

I'm use LXD container with ubuntu 18.04 with limit to resources -- 13gb Ram and 4 core

Elasticsearch version 6.6.0

Configs JVM for Elastic
cat /etc/elasticsearch/jvm.options

-Xms4g
-Xmx4g

-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly

--Des.networkaddress.cache.ttl=60
-Des.networkaddress.cache.negative.ttl=10

-XX:+AlwaysPreTouch

-Xss1m

-Djava.awt.headless=true

-Dfile.encoding=UTF-8

-Djna.nosys=true

-XX:-OmitStackTraceInFastThrow

-Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true
-Dio.netty.recycler.maxCapacityPerThread=0

-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true

-Djava.io.tmpdir=${ES_TMPDIR}

-XX:+HeapDumpOnOutOfMemoryError

-XX:HeapDumpPath=/var/lib/elasticsearch

-XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log

8:-XX:+PrintGCDetails
8:-XX:+PrintGCDateStamps
8:-XX:+PrintTenuringDistribution
8:-XX:+PrintGCApplicationStoppedTime
8:-Xloggc:/var/log/elasticsearch/gc.log
8:-XX:+UseGCLogFileRotation
8:-XX:NumberOfGCLogFiles=32
8:-XX:GCLogFileSize=64m

9-:-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m
9-:-Djava.locale.providers=COMPAT

10-:-XX:UseAVX=2

cat /proc/meminfo

MemTotal:       13671872 kB
MemFree:         4857240 kB
MemAvailable:    6115816 kB
Buffers:               0 kB
Cached:          1258576 kB
SwapCached:            0 kB
Active:          8171968 kB
Inactive:         464476 kB
Active(anon):    7367932 kB
Inactive(anon):      124 kB
Active(file):     804036 kB
Inactive(file):   464352 kB
Unevictable:           0 kB
Mlocked:        33652304 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:              2772 kB
Writeback:             0 kB
AnonPages:      73933548 kB
Mapped:          3917012 kB
Shmem:              9500 kB
Slab:               0 kB
SReclaimable:          0 kB
SUnreclaim:            0 kB
KernelStack:      125056 kB
PageTables:       288648 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    157965700 kB
Committed_AS:   122171068 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:    512000 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:     2520896 kB
DirectMap2M:    109289472 kB
DirectMap1G:    24117248 kB

cat /proc/sys/vm/overcommit_ratio
120

cat /proc/sys/vm/overcommit_memory
0

2

That all looks fairly normal to me. What is your question?

3

why ES use 46637680 VSZ?
in the HTOP --- 44.5G VIRT

my heap size only 4Gb

-Xms4g
-Xmx4g

4

VSZ includes lots of things that aren't on the heap (or even in-memory) -- libraries, memory-mapped indices, and so on. It doesn't relate to the memory usage of the process, if that's what you are asking.

5

thank for answer

can i reduce this VSZ size used by ES?

6

Hello, 111266.
I've earlier posted one advice Is there a way to limit RES memory used by ES however I've found that it might be hard to get a point.

7

Not really, but I'm not really understanding why you want to do this. Virtual memory isn't really a limited resource.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.