Initial heap size [199229440] not equal to maximum heap size [3187671040]; this can cause resize pauses and prevents mlockall from locking the entire heap

Elastic Stack Elasticsearch
1

I have installed elasticsearch 6.1.3 and below is the log while starting the server. (i have set -Xms2g
-Xmx2g in jvm.options file) but still facing "initial heap size [199229440] not equal to maximum heap size [3187671040]; this can cause resize pauses and prevents mlockall from locking the entire heap".
Appreciate response from experts.

'findstr' is not recognized as an internal or external command,
operable program or batch file.
[2018-07-31T11:37:46,794][INFO ][o.e.n.Node ] [] initializing ...
[2018-07-31T11:37:46,889][INFO ][o.e.e.NodeEnvironment ] [XZiaC9C] using [1] data paths, mounts [[(C:)]], net usable_space [158.3gb], net total_space [237.3gb], types [NTFS]
[2018-07-31T11:37:46,889][INFO ][o.e.e.NodeEnvironment ] [XZiaC9C] heap size [2.6gb], compressed ordinary object pointers [true]
[2018-07-31T11:37:46,891][INFO ][o.e.n.Node ] node name [XZiaC9C] derived from node ID [XZiaC9CmTqSM8PSQIJmdDw]; set [node.name] to override
[2018-07-31T11:37:46,891][INFO ][o.e.n.Node ] version[6.1.3], pid[16392], build[af51318/2018-01-26T18:22:55.523Z], OS[Windows 10/10.0/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_181/25.181-b13]
[2018-07-31T11:37:46,892][INFO ][o.e.n.Node ] JVM arguments [-Delasticsearch, -Des.path.home=C:\Users\Imran\elasticsearch-6.1.3, -Des.path.conf=C:\Users\Imran\elasticsearch-6.1.3\config]
[2018-07-31T11:37:48,166][INFO ][o.e.p.PluginsService ] [XZiaC9C] loaded module [aggs-matrix-stats]
[2018-07-31T11:37:48,167][INFO ][o.e.p.PluginsService ] [XZiaC9C] loaded module [analysis-common]
[2018-07-31T11:37:48,167][INFO ][o.e.p.PluginsService ] [XZiaC9C] loaded module [ingest-common]
[2018-07-31T11:37:48,167][INFO ][o.e.p.PluginsService ] [XZiaC9C] loaded module [lang-expression]
[2018-07-31T11:37:48,168][INFO ][o.e.p.PluginsService ] [XZiaC9C] loaded module [lang-mustache]
[2018-07-31T11:37:48,168][INFO ][o.e.p.PluginsService ] [XZiaC9C] loaded module [lang-painless]
[2018-07-31T11:37:48,168][INFO ][o.e.p.PluginsService ] [XZiaC9C] loaded module [mapper-extras]
[2018-07-31T11:37:48,168][INFO ][o.e.p.PluginsService ] [XZiaC9C] loaded module [parent-join]
[2018-07-31T11:37:48,169][INFO ][o.e.p.PluginsService ] [XZiaC9C] loaded module [percolator]
[2018-07-31T11:37:48,169][INFO ][o.e.p.PluginsService ] [XZiaC9C] loaded module [reindex]
[2018-07-31T11:37:48,169][INFO ][o.e.p.PluginsService ] [XZiaC9C] loaded module [repository-url]
[2018-07-31T11:37:48,170][INFO ][o.e.p.PluginsService ] [XZiaC9C] loaded module [transport-netty4]
[2018-07-31T11:37:48,170][INFO ][o.e.p.PluginsService ] [XZiaC9C] loaded module [tribe]
[2018-07-31T11:37:48,171][INFO ][o.e.p.PluginsService ] [XZiaC9C] no plugins loaded
[2018-07-31T11:37:48,359][INFO ][i.n.u.i.PlatformDependent] Your platform does not provide complete low-level API for accessing direct buffers reliably. Unless explicitly requested, heap buffer will always be preferred to avoid potential system instability.
[2018-07-31T11:37:50,074][INFO ][o.e.d.DiscoveryModule ] [XZiaC9C] using discovery type [zen]
[2018-07-31T11:37:50,752][INFO ][o.e.n.Node ] initialized
[2018-07-31T11:37:50,753][INFO ][o.e.n.Node ] [XZiaC9C] starting ...
[2018-07-31T11:37:51,415][INFO ][o.e.t.TransportService ] [XZiaC9C] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}, {[::1]:9300}
[2018-07-31T11:37:51,425][WARN ][o.e.b.BootstrapChecks ] [XZiaC9C] initial heap size [199229440] not equal to maximum heap size [3187671040]; this can cause resize pauses and prevents mlockall from locking the entire heap
[2018-07-31T11:37:54,485][INFO ][o.e.c.s.MasterService ] [XZiaC9C] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {XZiaC9C}{XZiaC9CmTqSM8PSQIJmdDw}{xzzn2HtZTvqY5kMZ42cqdg}{127.0.0.1}{127.0.0.1:9300}
[2018-07-31T11:37:54,489][INFO ][o.e.c.s.ClusterApplierService] [XZiaC9C] new_master {XZiaC9C}{XZiaC9CmTqSM8PSQIJmdDw}{xzzn2HtZTvqY5kMZ42cqdg}{127.0.0.1}{127.0.0.1:9300}, reason: apply cluster state (from master [master {XZiaC9C}{XZiaC9CmTqSM8PSQIJmdDw}{xzzn2HtZTvqY5kMZ42cqdg}{127.0.0.1}{127.0.0.1:9300} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[2018-07-31T11:37:54,516][INFO ][o.e.g.GatewayService ] [XZiaC9C] recovered [0] indices into cluster_state
[2018-07-31T11:37:54,818][INFO ][o.e.h.n.Netty4HttpServerTransport] [XZiaC9C] publish_address {127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}, {[::1]:9200}
[2018-07-31T11:37:54,818][INFO ][o.e.n.Node ] [XZiaC9C] started

2

can you share your configuration files as well?

Also, are you setting environment variables that potentially change or overwrite the heap size?

Judging from this line

[2018-07-31T11:37:46,889][INFO ][o.e.e.NodeEnvironment ] [XZiaC9C] heap size [2.6gb], compressed ordinary object pointers [true]

you heap seems to be configured not to those two gb, which means there must be some mechanism to overwrite this. Can you check the jvm.options file for duplicate entries containing -Xms or -Xmx?

4

Hi Alexander,
Here is my jvm.options file

    ## JVM configuration

################################################################
## IMPORTANT: JVM heap size
################################################################
##
## You should always set the min and max JVM heap
## size to the same value. For example, to set
## the heap to 4 GB, set:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
## for more information
##
################################################################

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms2g 
-Xmx2g

################################################################
## Expert settings
################################################################
##
## All settings below this section are considered
## expert settings. Don't tamper with them unless
## you understand what you are doing
##
################################################################

## GC configuration
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly

## optimizations

# pre-touch memory pages used by the JVM during initialization
-XX:+AlwaysPreTouch

## basic

# force the server VM
-server

# explicitly set the stack size
-Xss1m

# set to headless, just in case
-Djava.awt.headless=true

# ensure UTF-8 encoding by default (e.g. filenames)
-Dfile.encoding=UTF-8

# use our provided JNA always versus the system one
-Djna.nosys=true

# turn off a JDK optimization that throws away stack traces for common
# exceptions because stack traces are important for debugging
-XX:-OmitStackTraceInFastThrow

# flags to configure Netty
-Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true
-Dio.netty.recycler.maxCapacityPerThread=0

# log4j 2
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true

## heap dumps

# generate a heap dump when an allocation from the Java heap fails
# heap dumps are created in the working directory of the JVM
-XX:+HeapDumpOnOutOfMemoryError

# specify an alternative path for heap dumps
# ensure the directory exists and has sufficient space
#-XX:HeapDumpPath=/heap/dump/path

## GC logging

#-XX:+PrintGCDetails
#-XX:+PrintGCTimeStamps
#-XX:+PrintGCDateStamps
#-XX:+PrintClassHistogram
#-XX:+PrintTenuringDistribution
#-XX:+PrintGCApplicationStoppedTime

# log GC status to a file with time stamps
# ensure the directory exists
#-Xloggc:${loggc}

# By default, the GC log file will not rotate.
# By uncommenting the lines below, the GC log file
# will be rotated every 128MB at most 32 times.
#-XX:+UseGCLogFileRotation
#-XX:NumberOfGCLogFiles=32
#-XX:GCLogFileSize=128M

Here below is my elasticsearch.yml file
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
#
# Path to log files:
#
#path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: 192.168.0.1
#
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes: 2
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#

Yes, I did set environmental variables like below
ES_JAVA_OPTS="-Xms2g -Xmx2g" ./bin/elasticsearch
ES_JAVA_OPTS="-Xms4000m -Xmx4000m" ./bin/elasticsearch

I could not find any duplicates either in jvm.options file and setting up environmental variables also doesn't work.

Your help to resolve this would be very helpful. Thanks in advance.Preformatted text

5

This line from your initial start up is interesting. It seems that the jvm options file is not parsed at all.

In that line all the parsed options from jvm.options are usually listed. This one here contains nothing. Can you fully unset JAVA_OPTS and ES_JAVA_OPTS, retry and check that line output. Are there any other environment variables that you have set?

This is the output on my local osx installation (much longer than yours)

[2018-08-03T09:02:54,601][INFO ][o.e.n.Node               ] [NlLsyIx] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/var/folders/xx/111r6kc974z_tmqc70rymxkm0000gn/T/elasticsearch.fDy2AbH4, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -XX:UseAVX=2, -Des.path.home=/Users/alr/Downloads/6.3.2/elasticsearch-6.3.2, -Des.path.conf=/Users/alr/Downloads/6.3.2/elasticsearch-6.3.2/config, -Des.distribution.flavor=default, -Des.distribution.type=zip]
6

Please find attached.

image
image.png1920×1080 287 KB
.
more interestingly there is no path like below
Des.path.home=/Users/alr/Downloads/6.3.2/elasticsearch-6.3.2, -Des.path.conf=/Users/alr/Downloads/6.3.2/elasticsearch-6.3.2/config.

Please suggest, if you want me to unset these and run server.

7

Please don't post images of text as they are hardly readable and not searchable.

Instead paste the text and format it with </> icon. Check the preview window.

1 Like
8 
[2018-08-03T09:02:54,601][INFO ][o.e.n.Node               ] 
[NlLsyIx] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, 
-XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, 
-Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, 
-Djava.io.tmpdir=/var/folders/xx/111r6kc974z_tmqc70rymxkm0000gn/T/elasticsearch.fDy2AbH4, 
-XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, 
-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, 
-Djava.locale.providers=COMPAT, -XX:UseAVX=2, -Des.path.home=/Users/alr/Downloads/6.3.2/elasticsearch-6.3.2,
-Des.path.conf=/Users/alr/Downloads/6.3.2/elasticsearch-6.3.2/config, -Des.distribution.flavor=default, -Des.distribution.type=zip]
The filename, directory name, or volume label syntax is incorrect.
9

now your last message shows, that elasticsearch has started with a 2gb heap, as you wanted it to start, correct? The jvm.options file seems to have been properly parsed now

10

Yes. It is not showing in elasticsearch server log and able to see in environment variables but as i mentioned, there is no path like below as showing in image
Des.path.home=/Users/alr/Downloads/6.3.2/elasticsearch-6.3.2, -Des.path.conf=/Users/alr/Downloads/6.3.2/elasticsearch-6.3.2/config.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.