Hello,
I have an elasticsearch cluster with 4 nodes, two process on each node(listen 8200 and 9200 port).
Now I have a index with 8 shards and 1 replications. but the index always exsit an unassigned shard. Before ask the question, I have POST /_cluster/reroute?retry_failed=true' or set {"number_of_replicas":0} and back to {"number_of_replicas":1} for many times.but still not work. the cluster is still yellow.
below is the index configs and /_cluster/allocation/explain result.
It tells me one node can not connect,And I am sure the port listen of node is normal,becuase there are 2 primary shards on this node.
curl -s 'localhost:9200/adservice1_20190321/_settings?pretty'
{
"adservice1_20190321" : {
"settings" : {
"index" : {
"codec" : "best_compression",
"routing" : {
"allocation" : {
"require" : {
"rack" : "stale"
}
}
},
"search" : {
"slowlog" : {
"threshold" : {
"fetch" : {
"warn" : "1s",
"trace" : "200ms",
"debug" : "500ms",
"info" : "800ms"
},
"query" : {
"warn" : "10s",
"trace" : "500ms",
"debug" : "2s",
"info" : "5s"
}
}
}
},
"refresh_interval" : "60s",
"number_of_shards" : "8",
"translog" : {
"sync_interval" : "5s",
"durability" : "async"
},
"provided_name" : "adservice1_20190321",
"merge" : {
"scheduler" : {
"max_thread_count" : "2"
}
},
"creation_date" : "1553212811542",
"number_of_replicas" : "1",
"uuid" : "TAngC1U3R8qneD-dnbMgNA",
"version" : {
"created" : "5020299"
}
}
}
}
}{
"index" : "adservice1_20190321",
"shard" : 3,
"primary" : false,
"current_state" : "unassigned",
"unassigned_info" : {
"reason" : "ALLOCATION_FAILED",
"at" : "2019-03-21T01:44:50.171Z",
"failed_allocation_attempts" : 7,
"details" : "failed to perform indices:data/write/bulk[s] on replica [adservice1_20190321][3], node[nlTM3tPtSLKn7vJItD0gqw], [R], recovery_source[peer recovery], s[INITIALIZING], a[id=X0fb0yW0QRWZs_OwdDZPag], unassigned_info[[reason=ALLOCATION_FAILED], at[2019-03-21T01:44:34.951Z], failed_attempts[6], delayed=false, details[failed to perform indices:data/write/bulk[s] on replica [adservice1_20190321][3], node[nlTM3tPtSLKn7vJItD0gqw], [R], recovery_source[peer recovery], s[INITIALIZING], a[id=-Mr1-CWPR7O2giG4cHBuHQ], unassigned_info[[reason=ALLOCATION_FAILED], at[2019-03-21T00:01:11.615Z], failed_attempts[5], delayed=false, details[failed to perform indices:data/write/bulk[s] on replica [adservice1_20190321][3], node[nlTM3tPtSLKn7vJItD0gqw], [R], recovery_source[peer recovery], s[INITIALIZING], a[id=_qxUN09-TdmBmkQDFAKjZQ], unassigned_info[[reason=ALLOCATION_FAILED], at[2019-03-21T00:01:10.202Z], failed_attempts[4], delayed=false, details[failed to perform indices:data/write/bulk[s] on replica [adservice1_20190321][3], node[x9q2x22EQMi_PlQiau552w], [R], recovery_source[peer recovery], s[INITIALIZING], a[id=Eo4-nMHETUG4ReGUYi5-ng], unassigned_info[[reason=ALLOCATION_FAILED], at[2019-03-21T00:01:08.793Z], failed_attempts[3], delayed=false, details[failed to perform indices:data/write/bulk[s] on replica [adservice1_20190321][3], node[nlTM3tPtSLKn7vJItD0gqw], [R], recovery_source[peer recovery], s[INITIALIZING], a[id=7ZJ2I8XzTWmlPXwPM_fKWw], unassigned_info[[reason=ALLOCATION_FAILED], at[2019-03-21T00:01:01.983Z], failed_attempts[2], delayed=false, details[failed to perform indices:data/write/bulk[s] on replica [adservice1_20190321][3], node[x9q2x22EQMi_PlQiau552w], [R], recovery_source[peer recovery], s[INITIALIZING], a[id=_vEVLtoZQFCeAY8bg_fK-w], unassigned_info[[reason=ALLOCATION_FAILED], at[2019-03-21T00:00:58.817Z], failed_attempts[1], delayed=false, details[failed to perform indices:data/write/bulk[s] on replica [adservice1_20190321][3], node[nlTM3tPtSLKn7vJItD0gqw], [R], s[STARTED], a[id=xqjVvuU9TzKxmjp2o5v-Pg], failure NodeDisconnectedException[[node1-10.226.154.25][10.226.154.25:9400][indices:data/write/bulk[s][r]] disconnected]], allocation_status[no_attempt]], expected_shard_size[130], failure NodeDisconnectedException[[node1-10.226.137.176][10.226.137.176:9400][indices:data/write/bulk[s][r]] disconnected]], allocation_status[no_attempt]], expected_shard_size[130], failure NodeDisconnectedException[[node1-10.226.154.25][10.226.154.25:9400][indices:data/write/bulk[s][r]] disconnected]], allocation_status[no_attempt]], expected_shard_size[130], failure NodeDisconnectedException[[node1-10.226.137.176][10.226.137.176:9400][indices:data/write/bulk[s][r]] disconnected]], allocation_status[no_attempt]], expected_shard_size[130], failure NodeDisconnectedException[[node1-10.226.154.25][10.226.154.25:9400][indices:data/write/bulk[s][r]] disconnected]], allocation_status[no_attempt]], failure NodeDisconnectedException[[node1-10.226.154.25][10.226.154.25:9400][indices:data/write/bulk[s][r]] disconnected]], allocation_status[no_attempt]], failure NodeDisconnectedException[[node1-10.226.154.25][10.226.154.25:9400][indices:data/write/bulk[s][r]] disconnected]",
"last_allocation_status" : "no_attempt"
},
"can_allocate" : "no",
"allocate_explanation" : "cannot allocate because allocation is not permitted to any of the nodes",somebody can help me? thanks very much.