Hello I have a new data node added to the cluster, when I am trying to relocate some shards to it, i get:
{
"error": {
"root_cause": [
{
"type": "remote_transport_exception",
"reason": "[lwes-search-master1][xx.x.xx.xxx:9300][cluster:admin/reroute]"
}
],
"type": "illegal_argument_exception",
"reason": "[move_allocation] can't move [paloalto-2016.08.22][4], from {prod526}{Cgao_cOiTJKqlibtxHcmlQ}{xx.xxx.xxx.xx.}{xxx.xx.xx.xx9300}{max_local_storage_nodes=1, master=false}, to {prod524}{SfhBjFBxRaCyeDgDtbrIqg}{xxx.x.xx.xx}{xx.xxx.xx.xxx:9300}{max_local_storage_nodes=1, master=false}, since its not allowed, reason: [YES(allocation disabling is ignored)][YES(allocation disabling is ignored)][NO(after allocation less than required [0b] free on node, free: [338.6mb])][YES(node passes include/exclude/require filters)][YES(shard not primary or relocation disabled)][YES(below shard recovery limit of [2])][YES(shard is not allocated to same node or host)][YES(total shard limit disabled: [index: -1, cluster: -1] <= 0)][YES(target node version [2.1.2] is same or newer than source node version [2.1.2])][YES(shard is primary)][YES(no allocation awareness enabled)]"
},
"status": 400
}
These are my cluster settings:
curl -XGET localhost:9200/_cluster/settings?pretty
{
"persistent" : {
"threadpool" : {
"bulk" : {
"queue_size" : "100"
},
"index" : {
"queue_size" : "500"
}
}
},
"transient" : {
"cluster" : {
"routing" : {
"allocation" : {
"disable_allocation" : "false",
"disable_replica_allocation" : "false",
"disk" : {
"watermark" : {
"low" : "85%",
"high" : "200gb"
}
},
"enable" : "all"
}
},
"info" : {
"update" : {
"interval" : "1m"
}
}
}
}
}
my data path is pointing to a partition that has over 3.0T space, and has proper permissions