When add new node in elasticsearch 5.4 old node deleted - 2


(Jayram) #1

Hello

I created a node via command as elasticsearch user in server

here is command

bin/elasticsearch -Epath.data=/usr/share/elasticsearch/data/node_3 -Epath.logs=/var/log/elasticsearch/node_3 -Enode.name=fan

then it's showing fine

then i add another via command

bin/elasticsearch -Epath.data=/usr/share/elasticsearch/data/node_4 -Epath.logs=/var/log/elasticsearch/node_4 -Enode.name=fan_1

then fan is remove when i fire query like this

curl -XGET 'http://localhost:9200/_cluster/state?pretty'

Here is Log

[2017-05-19T13:20:27,612][INFO ][o.e.n.Node               ] [paresh] initializing ...
[2017-05-19T13:20:27,715][INFO ][o.e.e.NodeEnvironment    ] [paresh] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [52.9gb], net total_space [59gb], spins? [unknown], types [rootfs]
[2017-05-19T13:20:27,715][INFO ][o.e.e.NodeEnvironment    ] [paresh] heap size [247.5mb], compressed ordinary object pointers [true]
[2017-05-19T13:20:27,718][INFO ][o.e.n.Node               ] [paresh] node name [paresh], node ID [oDsesoB-QwCcKEAlWbytzw]
[2017-05-19T13:20:27,719][INFO ][o.e.n.Node               ] [paresh] version[5.4.0], pid[21524], build[780f8c4/2017-04-28T17:43:27.229Z], OS[Linux/3.10.0-514.16.1.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_131/25.131-b11]
[2017-05-19T13:20:29,029][INFO ][o.e.p.PluginsService     ] [paresh] loaded module [aggs-matrix-stats]
[2017-05-19T13:20:29,029][INFO ][o.e.p.PluginsService     ] [paresh] loaded module [ingest-common]
[2017-05-19T13:20:29,029][INFO ][o.e.p.PluginsService     ] [paresh] loaded module [lang-expression]
[2017-05-19T13:20:29,029][INFO ][o.e.p.PluginsService     ] [paresh] loaded module [lang-groovy]
[2017-05-19T13:20:29,030][INFO ][o.e.p.PluginsService     ] [paresh] loaded module [lang-mustache]
[2017-05-19T13:20:29,030][INFO ][o.e.p.PluginsService     ] [paresh] loaded module [lang-painless]
[2017-05-19T13:20:29,030][INFO ][o.e.p.PluginsService     ] [paresh] loaded module [percolator]
[2017-05-19T13:20:29,030][INFO ][o.e.p.PluginsService     ] [paresh] loaded module [reindex]
[2017-05-19T13:20:29,030][INFO ][o.e.p.PluginsService     ] [paresh] loaded module [transport-netty3]
[2017-05-19T13:20:29,030][INFO ][o.e.p.PluginsService     ] [paresh] loaded module [transport-netty4]
[2017-05-19T13:20:29,031][INFO ][o.e.p.PluginsService     ] [paresh] no plugins loaded
[2017-05-19T13:20:30,869][INFO ][o.e.d.DiscoveryModule    ] [paresh] using discovery type [zen]
[2017-05-19T13:20:31,708][INFO ][o.e.n.Node               ] [paresh] initialized
[2017-05-19T13:20:31,708][INFO ][o.e.n.Node               ] [paresh] starting ...
[2017-05-19T13:20:32,092][INFO ][o.e.t.TransportService   ] [paresh] publish_address {127.0.0.1:9302}, bound_addresses {[::1]:9302}, {127.0.0.1:9302}
[2017-05-19T13:20:35,472][INFO ][o.e.c.s.ClusterService   ] [paresh] detected_master {node_1}{44DUR4IFSQStThQWZoGpzg}{05XVwjEDRzumU88ojR-weQ}{127.0.0.1}{127.0.0.1:9300}, added {{fangels}{n0UBDKQOTIanjtVUL_sPlQ}{eYkGb8VkRna0Zsw082ZZYg}{127.0.0.1}{127.0.0.1:9301},{node_1}{44DUR4IFSQStThQWZoGpzg}{05XVwjEDRzumU88ojR-weQ}{127.0.0.1}{127.0.0.1:9300},}, reason: zen-disco-receive(from master [master {node_1}{44DUR4IFSQStThQWZoGpzg}{05XVwjEDRzumU88ojR-weQ}{127.0.0.1}{127.0.0.1:9300} committed version [6]])
[2017-05-19T13:21:02,191][WARN ][o.e.n.Node               ] [paresh] timed out while waiting for initial discovery state - timeout: 30s
[2017-05-19T13:21:02,231][INFO ][o.e.h.n.Netty4HttpServerTransport] [paresh] publish_address {127.0.0.1:9202}, bound_addresses {[::1]:9202}, {127.0.0.1:9202}
[2017-05-19T13:21:02,243][INFO ][o.e.n.Node               ] [paresh] started
[2017-05-19T13:21:05,501][WARN ][o.e.c.NodeConnectionsService] [paresh] failed to connect to node {fangels}{n0UBDKQOTIanjtVUL_sPlQ}{eYkGb8VkRna0Zsw082ZZYg}{127.0.0.1}{127.0.0.1:9301} (tried [1] times)
org.elasticsearch.transport.ConnectTransportException: [fangels][127.0.0.1:9301] handshake_timeout[30s]
	at org.elasticsearch.transport.TcpTransport.executeHandshake(TcpTransport.java:1589) ~[elasticsearch-5.4.0.jar:5.4.0]
	at org.elasticsearch.transport.TcpTransport.openConnection(TcpTransport.java:533) ~[elasticsearch-5.4.0.jar:5.4.0]
	at org.elasticsearch.transport.TcpTransport.connectToNode(TcpTransport.java:465) ~[elasticsearch-5.4.0.jar:5.4.0]
	at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:315) ~[elasticsearch-5.4.0.jar:5.4.0]
	at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:302) ~[elasticsearch-5.4.0.jar:5.4.0]
	at org.elasticsearch.cluster.NodeConnectionsService.validateAndConnectIfNeeded(NodeConnectionsService.java:154) [elasticsearch-5.4.0.jar:5.4.0]
	at org.elasticsearch.cluster.NodeConnectionsService$1.doRun(NodeConnectionsService.java:107) [elasticsearch-5.4.0.jar:5.4.0]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) [elasticsearch-5.4.0.jar:5.4.0]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.4.0.jar:5.4.0]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]
[2017-05-19T13:21:05,529][WARN ][o.e.c.s.ClusterService   ] [paresh] cluster state update task [zen-disco-receive(from master [master {node_1}{44DUR4IFSQStThQWZoGpzg}{05XVwjEDRzumU88ojR-weQ}{127.0.0.1}{127.0.0.1:9300} committed version [6]])] took [30s] above the warn threshold of 30s
[2017-05-19T13:21:28,213][INFO ][o.e.c.s.ClusterService   ] [paresh] removed {{fangels}{n0UBDKQOTIanjtVUL_sPlQ}{eYkGb8VkRna0Zsw082ZZYg}{127.0.0.1}{127.0.0.1:9301},}, reason: zen-disco-receive(from master [master {node_1}{44DUR4IFSQStThQWZoGpzg}{05XVwjEDRzumU88ojR-weQ}{127.0.0.1}{127.0.0.1:9300} committed version [7]])

what is missing please guide me


(Mark Walkom) #2

It would be better if you add this to your other thread - When add new node in elasticsearch 5.4 old node deleted


(Mark Walkom) #3