Elasticsearch v5.0.2 (master eligible) nodes as Docker container keeps on restarting

Hi all,

Pardon me if my issue has been answered somewhere or is not the right place to ask. This is my first post to this forum.

I built a custom Docker image based on RHEL v7.x using the Elasticsearch v5.0.2 RPM from Elastic and Oracle Server JRE v1.0.8_161. However the master eligible nodes as Docker container keeps on restarting.

Below is a sample Docker command I executed to run master eligible Docker container:

$ docker run -d --name master01 --hostname master01 \
--privileged=true --cap-add=IPC_LOCK --restart always \
--cpus 2.0 -m 2560M --memory-reservation 2048M --memory-swap=0 --memory-swappiness=0 \
--ulimit memlock=-1:-1 --ulimit nofile=131072:262144 --ulimit nproc=131072:262144 \
-e MALLOC_ARENA_MAX=4 -e environment=development -e cluster=elasticsearch -e cluster_type=transaction \
--add-host "coordinating01 coordinating01.example.com:192.168.210.15" \
--add-host "coordinating02 coordinating02.example.com:192.168.210.16" \
--add-host "coordinating03 coordinating03.example.com:192.168.210.17" \
--add-host "data01 data01.example.com:192.168.210.15" \
--add-host "data02 data02.example.com:192.168.210.16" \
--add-host "data03 data03.example.com:192.168.210.17" \
--add-host "ingest01 ingest01.example.com:192.168.210.15" \
--add-host "ingest02 ingest02.example.com:192.168.210.16" \
--add-host "ingest03 ingest03.example.com:192.168.210.17" \
--add-host "master01 master01.example.com:192.168.210.15" \
--add-host "master02 master02.example.com:192.168.210.16" \
--add-host "master03 master03.example.com:192.168.210.17" \
--label name=master01 --label environment=development \
-v /etc/localtime:/etc/localtime:ro \
-v /home/elasticsearch/nfs/common/elasticsearch/configs/master01.yml:/etc/elasticsearch/elasticsearch.yml:ro \
-v /home/elasticsearch/nfs/common/elasticsearch/configs/jvm.options-master:/etc/elasticsearch/jvm.options:ro \
-v /home/elasticsearch/nfs/common/elasticsearch/configs/log4j2.properties:/etc/elasticsearch/log4j2.properties:ro \
-v /home/elasticsearch/data/db/elasticsearch/master/01:/data \
-v /home/elasticsearch/nfs/data/db/elasticsearch/master/01:/app/snapshot \
-v /home/elasticsearch/nfs/data/log/elasticsearch/master/01:/var/log/elasticsearch \
-w /usr/share/elasticsearch --network host -p 9200:9200 -p 9300:9300 elasticsearch:1522210252 \
/usr/local/bin/elasticsearch -p /var/run/elasticsearch/elasticsearch.pid -Edefault.path.conf=/etc/elasticsearch \
-Etransport.publish_port=9300 -Ehttp.publish_port=9200

Thank you in advance.

Below are the current Elasticsearch master eligible node configurations:

[ master01.yml ]

cluster.name: ES_Cluster
node.name: master01
node.master: true
node.data: false
node.ingest: false
index.codec: best_compression
path.data: /data
path.logs: /var/log/elasticsearch
path.repo: ["/app/snapshot"]
bootstrap.memory_lock: true
network.host: _site_
transport.host: _site_
http.host: _site_
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["master01:9300", "master02:9300", "master03:9300"]
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping_timeout: 300s
discovery.zen.join_timeout: 150s
discovery.zen.publish_timeout: 150s
discovery.zen.fd.ping_interval: 150s
discovery.zen.fd.ping_timeout: 300s
discovery.zen.fd.ping_retries: 3
cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.low: .98
cluster.routing.allocation.disk.watermark.high: .99
xpack.security.enabled: false
xpack.monitoring.enabled: false
xpack.watcher.enabled: false

[ master02.yml ]

cluster.name: ES_Cluster
node.name: master02
node.master: true
node.data: false
node.ingest: false
index.codec: best_compression
path.data: /data
path.logs: /var/log/elasticsearch
path.repo: ["/app/snapshot"]
bootstrap.memory_lock: true
network.host: _site_
transport.host: _site_
http.host: _site_
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["master01:9300", "master02:9300", "master03:9300"]
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping_timeout: 300s
discovery.zen.join_timeout: 150s
discovery.zen.publish_timeout: 150s
discovery.zen.fd.ping_interval: 150s
discovery.zen.fd.ping_timeout: 300s
discovery.zen.fd.ping_retries: 3
cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.low: .98
cluster.routing.allocation.disk.watermark.high: .99
xpack.security.enabled: false
xpack.monitoring.enabled: false
xpack.watcher.enabled: false

[ master03.yml]

cluster.name: ES_Cluster
node.name: master03
node.master: true
node.data: false
node.ingest: false
index.codec: best_compression
path.data: /data
path.logs: /var/log/elasticsearch
path.repo: ["/app/snapshot"]
bootstrap.memory_lock: true
network.host: _site_
transport.host: _site_
http.host: _site_
http.port: 9200
transport.tcp.port: 9300
discovery.zen.ping.unicast.hosts: ["master01:9300", "master02:9300", "master03:9300"]
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping_timeout: 300s
discovery.zen.join_timeout: 150s
discovery.zen.publish_timeout: 150s
discovery.zen.fd.ping_interval: 150s
discovery.zen.fd.ping_timeout: 300s
discovery.zen.fd.ping_retries: 3
cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.low: .98
cluster.routing.allocation.disk.watermark.high: .99
xpack.security.enabled: false
xpack.monitoring.enabled: false
xpack.watcher.enabled: false

Below are the logs:

# rpm -qi elasticsearch
Name        : elasticsearch
Epoch       : 0
Version     : 5.0.2
Release     : 1
Architecture: noarch
Install Date: Wed Mar 28 12:11:11 2018
Group       : Application/Internet
Size        : 36525042
License     : 2009
Signature   : RSA/SHA512, Thu Nov 24 18:30:01 2016, Key ID d27d666cd88e42b4
Source RPM  : elasticsearch-5.0.2-1-src.rpm
Build Date  : Thu Nov 24 18:09:58 2016
Build Host  : packer-virtualbox-iso-1468515721
Relocations : /usr
Packager    : Elasticsearch
Vendor      : Elasticsearch
URL         : https://www.elastic.co/
Summary     : Elasticsearch is a distributed RESTful search engine built for the cloud. Reference documentation can be found at https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html and the 'Elasticsearch: The Definitive Guide' book can be found at https://www.elastic.co/guide/en/elasticsearch/guide/current/index.html
Description :
Elasticsearch subproject :distribution:rpm

$ java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)

$ uname -r
3.10.0-693.11.6.el7.x86_64

$ docker exec master01 ls -l /usr/java/jdk1.8.0_161/jre/lib/rt.jar
-rw-r--r--. 1 10 143 66309610 Dec 20 08:21 /usr/java/jdk1.8.0_161/jre/lib/rt.jar

$ docker exec master01 stat /usr/java/jdk1.8.0_161/jre/lib/rt.jar
 File: '/usr/java/jdk1.8.0_161/jre/lib/rt.jar'
 Size: 66309610       Blocks: 129512    IO Block: 4096  regular file
Device: fd0ch/64780d   Inode: 4271129    Links: 1
Access: (0644/-rw-r--r--) Uid: (  10/ UNKNOWN)  Gid: ( 143/ UNKNOWN)
Access: 2018-03-29 12:45:41.666187269 +0800
Modify: 2017-12-20 08:21:14.000000000 +0800
Change: 2018-03-28 12:25:02.744575141 +0800
 Birth: -

Snippet of the logs

[2018-03-29T13:06:39,261][INFO ][o.e.b.BootstrapCheck     ] [master01] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2018-03-29T13:06:39,553][WARN ][o.e.d.z.p.u.UnicastZenPing] [master01] [1] failed send ping to {#zen_unicast_2#}{Efv1VRQnRnKF8t42W3jqgw}{192.168.210.16}{192.168.210.16:9300}
java.lang.IllegalStateException: handshake failed with {#zen_unicast_2#}{Efv1VRQnRnKF8t42W3jqgw}{192.168.210.16}{192.168.210.16:9300}
        at org.elasticsearch.transport.TransportService.handshake(TransportService.java:364) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.transport.TransportService.connectToNodeLightAndHandshake(TransportService.java:339) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.transport.TransportService.connectToNodeLightAndHandshake(TransportService.java:313) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing$2.run(UnicastZenPing.java:419) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:458) [elasticsearch-5.0.2.jar:5.0.2]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_161]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_161]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
Caused by: org.elasticsearch.transport.NodeDisconnectedException: [][192.168.210.16:9300][internal:transport/handshake] disconnected
[2018-03-29T13:07:09,282][WARN ][o.e.n.Node               ] [master01] timed out while waiting for initial discovery state - timeout: 30s
[2018-03-29T13:07:09,293][INFO ][o.e.h.HttpServer         ] [master01] publish_address {192.168.210.15:9200}, bound_addresses {172.17.0.1:9200}, {172.19.0.1:9200}, {192.168.210.15:9200}
[2018-03-29T13:07:09,294][INFO ][o.e.n.Node               ] [master01] started
[2018-03-29T13:11:39,453][WARN ][o.e.d.z.ZenDiscovery     ] [master01] failed to connect to master [{tp-elasticsearch-service-master-03}{DFpIqUY2Rdy2ZoS_5CIeeA}{ZxiWrmpTReqXxBxxQ9tF2g}{192.168.210.17}{192.168.210.17:9300}], retrying...
org.elasticsearch.transport.ConnectTransportException: [tp-elasticsearch-service-master-03][192.168.210.17:9300] connect_timeout[30s]
        at org.elasticsearch.transport.netty4.Netty4Transport.connectToChannels(Netty4Transport.java:379) ~[?:?]
        at org.elasticsearch.transport.TcpTransport.connectToNode(TcpTransport.java:403) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.transport.TcpTransport.connectToNode(TcpTransport.java:377) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:285) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.discovery.zen.ZenDiscovery.joinElectedMaster(ZenDiscovery.java:468) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.discovery.zen.ZenDiscovery.innerJoinCluster(ZenDiscovery.java:415) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.discovery.zen.ZenDiscovery.access$3800(ZenDiscovery.java:86) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.discovery.zen.ZenDiscovery$JoinThreadControl$1.run(ZenDiscovery.java:1187) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:458) [elasticsearch-5.0.2.jar:5.0.2]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_161]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_161]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /192.168.210.17:9300
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:?]
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:?]
        at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:347) ~[?:?]
        at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340) ~[?:?]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:627) ~[?:?]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:513) ~[?:?]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:467) ~[?:?]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:437) ~[?:?]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873) ~[?:?]
        ... 1 more

...
...
...

[2018-03-29T13:12:22,512][INFO ][o.e.n.Node               ] [master01] started
[2018-03-29T13:15:26,750][DEBUG][o.e.a.a.c.h.TransportClusterHealthAction] [master01] no known master node, scheduling a retry
[2018-03-29T13:15:56,753][DEBUG][o.e.a.a.c.h.TransportClusterHealthAction] [master01] timed out while retrying [cluster:monitor/health] after failure (timeout [30s])
[2018-03-29T13:15:56,754][WARN ][r.suppressed             ] path: /_cluster/health, params: {pretty=true}
org.elasticsearch.discovery.MasterNotDiscoveredException
        at org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$5.onTimeout(TransportMasterNodeAction.java:214) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.cluster.ClusterStateObserver$ContextPreservingListener.onTimeout(ClusterStateObserver.java:350) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.cluster.ClusterStateObserver$ObserverClusterStateListener.onTimeout(ClusterStateObserver.java:240) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.cluster.service.ClusterService$NotifyTimeout.run(ClusterService.java:957) [elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:458) [elasticsearch-5.0.2.jar:5.0.2]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_161]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_161]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
/usr/java/jdk1.8.0_161/jre/lib/rt.jar: error reading zip file
/usr/java/jdk1.8.0_161/jre/lib/rt.jar: error reading zip file
/usr/java/jdk1.8.0_161/jre/lib/rt.jar: error reading zip file
[2018-03-29T15:19:25,044][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [master01] fatal error in thread [main], exiting
java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: java/io/InterruptedIOException
        at org.elasticsearch.indices.analysis.AnalysisModule.setupAnalyzers(AnalysisModule.java:290) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.indices.analysis.AnalysisModule.<init>(AnalysisModule.java:173) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.node.Node.<init>(Node.java:304) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.node.Node.<init>(Node.java:217) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:196) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:196) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:291) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:96) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.cli.Command.main(Command.java:62) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89) ~[elasticsearch-5.0.2.jar:5.0.2]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82) ~[elasticsearch-5.0.2.jar:5.0.2]
Caused by: java.lang.NoClassDefFoundError: java/io/InterruptedIOException
        at sun.misc.Resource.getBytes(Resource.java:124) ~[?:?]
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:462) ~[?:1.8.0_161]
        at java.net.URLClassLoader.access$100(URLClassLoader.java:73) ~[?:1.8.0_161]
        at java.net.URLClassLoader$1.run(URLClassLoader.java:368) ~[?:1.8.0_161]
        at java.net.URLClassLoader$1.run(URLClassLoader.java:362) ~[?:1.8.0_161]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_161]
        at java.net.URLClassLoader.findClass(URLClassLoader.java:361) ~[?:1.8.0_161]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_161]
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338) ~[?:?]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_161]
        ... 14 more
2018-03-29 15:19:25,055 main ERROR Unable to write to stream /var/log/elasticsearch/ES_Cluster.log for appender rolling: org.apache.logging.log4j.core.appender.AppenderLoggingException: Error writing to stream /var/log/elasticsearch/ES_Cluster.log
/usr/java/jdk1.8.0_161/jre/lib/rt.jar: error reading zip file
/usr/java/jdk1.8.0_161/jre/lib/rt.jar: error reading zip file
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "main"

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.