Hi All,
I have done my 6 node ELK cluster setup and created a test index.
but after i created a test index shards are not getting relocated to other node and cluster status is changing to "RED" and i am getting the below error , hence can someone please assist me on this. I have checked all the YML configuration and logs and everything looks good.
ERROR
{
"index": "testes2",
"shard": 3,
"primary": true,
"current_state": "unassigned",
"unassigned_info": {
"reason": "INDEX_CREATED",
"at": "2019-12-19T07:16:16.791Z",
"last_allocation_status": "no"
},
"can_allocate": "no",
"allocate_explanation": "cannot allocate because allocation is not permitted to any of the nodes",
"node_allocation_decisions": [
{
"node_id": "NgdWtAOZSO2awEU315Ko5A",
"node_name": "ateu5m-cbs-elk-es-vm1",
"transport_address": "172.17.3.199:9300",
"node_decision": "no",
"weight_ranking": 1,
"deciders": [
{
"decider": "enable",
"decision": "NO",
"explanation": "no allocations are allowed due to cluster setting [cluster.routing.allocation.enable=none]"
},
{
"decider": "disk_threshold",
"decision": "NO",
"explanation": "the node is above the high watermark cluster setting [cluster.routing.allocation.disk.watermark.high=10gb], having less than the minimum required [10gb] free space, actual free: [6.3gb]"
}
]
},
{
"node_id": "-9NrqQqaTSqs0_LO5pgONA",
"node_name": "ateu5m-cbs-elk-es-vm6",
"transport_address": "172.17.3.205:9300",
"node_decision": "no",
"weight_ranking": 2,
"deciders": [
{
"decider": "enable",
"decision": "NO",
"explanation": "no allocations are allowed due to cluster setting [cluster.routing.allocation.enable=none]"
},
{
"decider": "disk_threshold",
"decision": "NO",
"explanation": "the node is above the high watermark cluster setting [cluster.routing.allocation.disk.watermark.high=10gb], having less than the minimum required [10gb] free space, actual free: [6.3gb]"
}
]
},
{
"node_id": "M6m_eb5jT3aXS1le6hVjkw",
"node_name": "ateu5m-cbs-elk-es-vm5",
"transport_address": "172.17.3.198:9300",
"node_decision": "no",
"weight_ranking": 3,
"deciders": [
{
"decider": "enable",
"decision": "NO",
"explanation": "no allocations are allowed due to cluster setting [cluster.routing.allocation.enable=none]"
},
{
"decider": "disk_threshold",
"decision": "NO",
"explanation": "the node is above the high watermark cluster setting [cluster.routing.allocation.disk.watermark.high=10gb], having less than the minimum required [10gb] free space, actual free: [6.3gb]"
}
]
},
{
"node_id": "Ggeh-XEPRi6wS-YpMrJNeg",
"node_name": "ateu5m-cbs-elk-es-vm4",
"transport_address": "172.17.3.200:9300",
"node_decision": "no",
"weight_ranking": 4,
"deciders": [
{
"decider": "enable",
"decision": "NO",
"explanation": "no allocations are allowed due to cluster setting [cluster.routing.allocation.enable=none]"
},
{
"decider": "disk_threshold",
"decision": "NO",
"explanation": "the node is above the high watermark cluster setting [cluster.routing.allocation.disk.watermark.high=10gb], having less than the minimum required [10gb] free space, actual free: [6.3gb]"
}
]
},
{
"node_id": "zPl1fdyuRM6cfv_SSzUY6g",
"node_name": "ateu5m-cbs-elk-es-vm2",
"transport_address": "172.17.3.202:9300",
"node_decision": "no",
"weight_ranking": 5,
"deciders": [
{
"decider": "enable",
"decision": "NO",
"explanation": "no allocations are allowed due to cluster setting [cluster.routing.allocation.enable=none]"
},
{
"decider": "disk_threshold",
"decision": "NO",
"explanation": "the node is above the high watermark cluster setting [cluster.routing.allocation.disk.watermark.high=10gb], having less than the minimum required [10gb] free space, actual free: [6.3gb]"
}
]
},
{
"node_id": "e3F4LDawSqSlIaf-_HZjKw",
"node_name": "ateu5m-cbs-elk-es-vm3",
"transport_address": "172.17.3.203:9300",
"node_decision": "no",
"weight_ranking": 6,
"deciders": [
{
"decider": "enable",
"decision": "NO",
"explanation": "no allocations are allowed due to cluster setting [cluster.routing.allocation.enable=none]"
},
{
"decider": "disk_threshold",
"decision": "NO",
"explanation": "the node is above the high watermark cluster setting [cluster.routing.allocation.disk.watermark.high=10gb], having less than the minimum required [10gb] free space, actual free: [6.3gb]"
}
]
}
]
}
Below are few information about my environment.
Total node : 6
Version : 6.2.4
Diskspace :
Filesystem Size Used Avail Use% Mounted on
devtmpfs 3.4G 0 3.4G 0% /dev
tmpfs 3.4G 0 3.4G 0% /dev/shm
tmpfs 3.4G 9.1M 3.4G 1% /run
tmpfs 3.4G 0 3.4G 0% /sys/fs/cgroup
/dev/mapper/rootvg-rootlv 7.8G 1014M 6.4G 14% /
/dev/mapper/rootvg-usrlv 9.8G 2.1G 7.2G 23% /usr
/dev/sda1 976M 44M 866M 5% /boot
/dev/mapper/rootvg-varlv 7.8G 222M 7.2G 3% /var
/dev/mapper/rootvg-homelv 976M 281M 629M 31% /home
/dev/mapper/rootvg-optlv 2.0G 6.1M 1.8G 1% /opt
/dev/mapper/rootvg-tmplv 2.0G 6.4M 1.8G 1% /tmp
/dev/sdb1 99G 61M 94G 1% /mnt/resource
/dev/mapper/vg_dynatrace-lv_dynatrace 2.9G 4.6M 2.8G 1% /dynatrace
tmpfs 696M 0 696M 0% /run/user/1000
Regards,
Santosh