elasticsearch-5.2.2-1.noarch yellow with shards stuck initializing ：
"initializing_shards" : 4,
"unassigned_shards" : 26,
http://10.25.0.14:20042/_cluster/health?pretty
{
"cluster_name" : "lc-es-cluster",
"status" : "yellow",
"timed_out" : false,
"number_of_nodes" : 4,
"number_of_data_nodes" : 3,
"active_primary_shards" : 85,
"active_shards" : 140,
"relocating_shards" : 0,
"initializing_shards" : 4,
"unassigned_shards" : 26,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 82.35294117647058
}
SETTING LIKE THIS:
cat /etc/elasticsearch/elasticsearch.yml | grep -v ^#
cluster.name: lc-es-cluster
node.name: node-172.19.1.31
node.master: true
node.data: false
node.attr.box_type: warm
bootstrap.memory_lock: true
bootstrap.system_call_filter: false
network.host: 0.0.0.0
network.publish_host: 172.19.1.31
http.port: 20042
transport.tcp.port: 20041
discovery.zen.ping.unicast.hosts: [172.19.1.31, 172.19.1.33, 172.19.1.34, 172.19.1.35]
discovery.zen.minimum_master_nodes: 2
indices.memory.index_buffer_size: 50%
thread_pool.bulk.queue_size: 8192
indices.fielddata.cache.size: 20%
JVM version (java -version):
java -version
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (build 1.8.0_121-b13)
OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)
OS version (uname -a if on a Unix-like system):
uname -a
Linux qa2-controller 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
Provide logs (if relevant):
and, these are some WARN info in the logs:
[2017-06-14T09:42:22,304][WARN ][o.e.c.a.s.ShardStateAction] [node-172.19.1.31]
[dfi_flow_pkt_hdr_batch__0_2687x1d_17061408][4] received shard failed for shard id
[[dfi_flow_pkt_hdr_batch__0_2687x1d_17061408][4]], allocation id [D6BpiMMRSuGN-r9Wz5oCvA],
primary term [1], message [failed to perform indices:data/write/bulk[s] on replica
[dfi_flow_pkt_hdr_batch__0_2687x1d_17061408][4], node[SEcmqqMpTw6bqNycpMSQ-A], [R],
recovery_source[peer recovery], s[INITIALIZING], a[id=D6BpiMMRSuGN-r9Wz5oCvA],
unassigned_info[[reason=INDEX_CREATED], at[2017-06-14T00:00:02.667Z], delayed=false,
allocation_status[no_attempt]]], failure [NodeDisconnectedException[[node-172.19.1.33][172.19.1.33:20041]
[indices:data/write/bulk[s][r]] disconnected]]
org.elasticsearch.transport.NodeDisconnectedException: [node-172.19.1.33][172.19.1.33:20041]
[indices:data/write/bulk[s][r]] disconnected
So, What I will do to resolve this issue?