Noticed that elasticsearach.log provide the info below:
[2020-05-05T15:55:48,241][WARN ][o.e.b.Natives ] [xxxxxxxxxxxxxxx] unable to load JNA native support library, native methods will be disabled.
java.lang.UnsatisfiedLinkError: /tmp/elasticsearch-7725357924971700307/jna--1985354563/jna2482199391639058917.tmp: /tmp/elasticsearch-7725357924971700307/jna--1985354563/jna2482199391639058917.tmp: failed to map segment from shared object: Operation not permitted
at java.lang.ClassLoader$NativeLibrary.load0(Native Method) ~[?:?]
at java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2440) ~[?:?]
at java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2497) ~[?:?]
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2694) ~[?:?]
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:2627) ~[?:?]
at java.lang.Runtime.load0(Runtime.java:744) ~[?:?]
at java.lang.System.load(System.java:1873) ~[?:?]
at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:947) ~[jna-4.5.1.jar:4.5.1 (b0)]
at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:922) ~[jna-4.5.1.jar:4.5.1 (b0)]
at com.sun.jna.Native.<clinit>(Native.java:190) ~[jna-4.5.1.jar:4.5.1 (b0)]
at java.lang.Class.forName0(Native Method) ~[?:?]
at java.lang.Class.forName(Class.java:333) ~[?:?]
at org.elasticsearch.bootstrap.Natives.<clinit>(Natives.java:45) [elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:104) [elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:172) [elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) [elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) [elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) [elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) [elasticsearch-cli-7.6.2.jar:7.6.2]
at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-7.6.2.jar:7.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) [elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-7.6.2.jar:7.6.2]
[2020-05-05T15:55:48,249][WARN ][o.e.b.Natives ] [xxxxxxxxxxxxxxx] cannot check if running as root because JNA is not available
[2020-05-05T15:55:48,249][WARN ][o.e.b.Natives ] [xxxxxxxxxxxxxxx] cannot install system call filter because JNA is not available
[2020-05-05T15:55:48,250][WARN ][o.e.b.Natives ] [xxxxxxxxxxxxxxx] cannot register console handler because JNA is not available
[2020-05-05T15:55:48,251][WARN ][o.e.b.Natives ] [xxxxxxxxxxxxxxx] cannot getrlimit RLIMIT_NPROC because JNA is not available
[2020-05-05T15:55:48,251][WARN ][o.e.b.Natives ] [xxxxxxxxxxxxxxx] cannot getrlimit RLIMIT_AS because JNA is not available
[2020-05-05T15:55:48,252][WARN ][o.e.b.Natives ] [xxxxxxxxxxxxxxx] cannot getrlimit RLIMIT_FSIZE because JNA is not available
[2020-05-05T15:55:48,419][INFO ][o.e.e.NodeEnvironment ] [xxxxxxxxxxxxxxx] using [1] data paths, mounts [[/var (/dev/mapper/osvg-var)]], net usable_space [3gb], net total_space [4.6gb], types [xfs]
[2020-05-05T15:55:48,420][INFO ][o.e.e.NodeEnvironment ] [xxxxxxxxxxxxxxx] heap size [990.7mb], compressed ordinary object pointers [true]
[2020-05-05T15:55:48,463][INFO ][o.e.n.Node ] [xxxxxxxxxxxxxxx] node name [xxxxxxxxxxxxxxx], node ID [racaeHmzTumhXPtCsFK0QQ], cluster name [elasticsearch]
[2020-05-05T15:55:48,464][INFO ][o.e.n.Node ] [xxxxxxxxxxxxxxx] version[7.6.2], pid[93865], build[default/rpm/ef48eb35cf30adf4db14086e8aabd07ef6fb113f/2020-03-26T06:34:37.794943Z], OS[Linux/3.10.0-1062.18.1.el7.x86_64/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/13.0.2/13.0.2+8]
[2020-05-05T15:55:48,464][INFO ][o.e.n.Node ] [xxxxxxxxxxxxxxx] JVM home [/usr/share/elasticsearch/jdk]
[2020-05-05T15:55:48,464][INFO ][o.e.n.Node ] [xxxxxxxxxxxxxxx] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=COMPAT, -Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.io.tmpdir=/tmp/elasticsearch-7725357924971700307, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -XX:MaxDirectMemorySize=536870912, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=default, -Des.distribution.type=rpm, -Des.bundled_jdk=true]
.
.
.
.
[2020-05-05T15:55:53,606][INFO ][o.e.x.s.a.s.FileRolesStore] [xxxxxxxxxxxxxxx] parsed [0] roles from file [/etc/elasticsearch/roles.yml]
[2020-05-05T15:55:54,050][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [xxxxxxxxxxxxxxx] [controller/93967] [Main.cc@110] controller (64 bit): Version 7.6.2 (Build e06ef9d86d5332) Copyright (c) 2020 Elasticsearch BV
[2020-05-05T15:55:54,568][DEBUG][o.e.a.ActionModule ] [xxxxxxxxxxxxxxx] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2020-05-05T15:55:54,695][INFO ][o.e.d.DiscoveryModule ] [xxxxxxxxxxxxxxx] using discovery type [zen] and seed hosts providers [settings]
[2020-05-05T15:55:55,570][INFO ][o.e.n.Node ] [xxxxxxxxxxxxxxx] initialized
[2020-05-05T15:55:55,571][INFO ][o.e.n.Node ] [xxxxxxxxxxxxxxx] starting ...
[2020-05-05T15:55:55,704][INFO ][o.e.t.TransportService ] [xxxxxxxxxxxxxxx] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}
[2020-05-05T15:55:55,935][WARN ][o.e.b.BootstrapChecks ] [xxxxxxxxxxxxxxx] system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
[2020-05-05T15:55:55,935][WARN ][o.e.b.BootstrapChecks ] [xxxxxxxxxxxxxxx] the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured
[2020-05-05T15:55:55,945][INFO ][o.e.c.c.ClusterBootstrapService] [xxxxxxxxxxxxxxx] no discovery configuration found, will perform best-effort cluster bootstrapping after [3s] unless existing master is discovered
[2020-05-05T15:55:58,952][INFO ][o.e.c.c.Coordinator ] [xxxxxxxxxxxxxxx] setting initial configuration to VotingConfiguration{racaeHmzTumhXPtCsFK0QQ}
[2020-05-05T15:55:59,203][INFO ][o.e.c.s.MasterService ] [xxxxxxxxxxxxxxx] elected-as-master ([1] nodes joined)[{xxxxxxxxxxxxxxx}{racaeHmzTumhXPtCsFK0QQ}{p5E3EM_OSwOJhgY7RUqUGA}{127.0.0.1}{127.0.0.1:9300}{dilm}{ml.machine_memory=16806420480, xpack.installed=true, ml.max_open_jobs=20} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 1, version: 1, delta: master node changed {previous [], current [{xxxxxxxxxxxxxxx}{racaeHmzTumhXPtCsFK0QQ}{p5E3EM_OSwOJhgY7RUqUGA}{127.0.0.1}{127.0.0.1:9300}{dilm}{ml.machine_memory=16806420480, xpack.installed=true, ml.max_open_jobs=20}]}
[2020-05-05T15:55:59,241][INFO ][o.e.c.c.CoordinationState] [xxxxxxxxxxxxxxx] cluster UUID set to [9fzd7MjoRDyJkTAEilDotQ]
[2020-05-05T15:55:59,263][INFO ][o.e.c.s.ClusterApplierService] [xxxxxxxxxxxxxxx] master node changed {previous [], current [{xxxxxxxxxxxxxxx}{racaeHmzTumhXPtCsFK0QQ}{p5E3EM_OSwOJhgY7RUqUGA}{127.0.0.1}{127.0.0.1:9300}{dilm}{ml.machine_memory=16806420480, xpack.installed=true, ml.max_open_jobs=20}]}, term: 1, version: 1, reason: Publication{term=1, version=1}
[2020-05-05T15:55:59,307][INFO ][o.e.h.AbstractHttpServerTransport] [xxxxxxxxxxxxxxx] publish_address {127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}
[2020-05-05T15:55:59,307][INFO ][o.e.n.Node ] [xxxxxxxxxxxxxxx] started
[2020-05-05T15:55:59,308][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [xxxxxxxxxxxxxxx] fatal error in thread [main], exiting
java.lang.NoClassDefFoundError: Could not initialize class com.sun.jna.Native
at org.elasticsearch.systemd.Libsystemd.lambda$static$0(Libsystemd.java:34) ~[?:?]
at java.security.AccessController.doPrivileged(AccessController.java:312) ~[?:?]
at org.elasticsearch.systemd.Libsystemd.<clinit>(Libsystemd.java:33) ~[?:?]
at org.elasticsearch.systemd.SystemdPlugin.sd_notify(SystemdPlugin.java:114) ~[?:?]
at org.elasticsearch.systemd.SystemdPlugin.onNodeStarted(SystemdPlugin.java:124) ~[?:?]
at java.util.ArrayList.forEach(ArrayList.java:1507) ~[?:?]
at org.elasticsearch.node.Node.start(Node.java:787) ~[elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:273) ~[elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:358) ~[elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) ~[elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) ~[elasticsearch-cli-7.6.2.jar:7.6.2]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.6.2.jar:7.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) ~[elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.6.2.jar:7.6.2]
I've pointed the temp java directory as below:
-Djava.io.tmpdir=/data1/els_dc1_1/tmp
For the mount on RHEL:
/dev/mapper/vgdb-data1 on /data1 type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
For /etc/fstab/ on RHEL:
/dev/mapper/vgdb-data1 /data1 xfs defaults 0 0
For the custom tmp directory /data1/els_dc1_1/tmp on RHEL:
drwxrwxrwx. 3 appadmin elasticsearch 29 May 6 02:29 tmp