Unassigned shards delaying allocation

Elastic Stack Elasticsearch
1

Hi guys!
I have a ES cluster with 6 nodes,
today two of these nodes was down, hosts were rebooted
After that i see a lot of uassigned shards in KOPF.

Screenshot_3.png1891×68 6.19 KB

and a lot of messages in logs:

>  [2015-10-15 11:55:12,765][INFO ][cluster.routing          ] [master] delaying allocation for [0] unassigned shards, next check in [0s]

also i want to say that three nodes havenot space on disk, they are in readonly

[2015-10-15 11:53:49,270][INFO ][cluster.routing.allocation.decider] [master] high disk watermark exceeded on one or more nodes, rerouting shards
[2015-10-15 11:54:49,253][WARN ][cluster.routing.allocation.decider] [master] high disk watermark [20gb] exceeded on [N9Z7j6TbQjaSdIkBIHoSgw][data3] free: 18gb[0.8%], shards will be relocated away from this node

What can I do to resolve this problem?
Thanks!

2

The hosts that were rebooted, are shards not being allocated on them? Is the low disk space condition on the other nodes?

3

Now all cluster looks green, strange...
In 4 hours there were unassigned shards...
Where can I trace what was happened?