Cluster health becomes RED after create a new index

Elastic Stack Elasticsearch
1

hi,
I met a very strange issue. My cluster (ES 1.7.1) always enters red status if I create a new index with only one doc.

Repro steps:

  1. get current health:
    http://localhost:9200/_cat/health

    1458814224 10:10:24 my-es-cluster green 12 6 36 36 0 0 0 0

    It is green.

  2. create a new index by posting a doc:

    http://localhost:9200/testindex/doc/1
    {
    "id": "1"
    }

    then got following response:

"error": "UnavailableShardsException[[testindex][2] Primary shard is not active or isn't assigned to a known node. Timeout: [1m], request: delete {[testindex][doc][1]}]"

Also, the cluster health becomes "red"

I checked the ES log file, only found following error message:
[2016-03-24 10:04:40,866][DEBUG][action.index ] [data-node-vm5] observer: timeout notification from cluster service. timeout setting [1m], time since start [1m]

How to troubleshoot the root cause?

2

Hi,

May be that you should assign "by hand" your shard.
GET _cat/shards

And check if this setting is correctly setted
cluster.routing.allocation.enable

More here https://www.elastic.co/guide/en/elasticsearch/reference/current/shards-allocation.html

3

Thanks iliasse!
You are right, the cluster.routing.allocation.enable was set to "none" accidentally by someone.
After changing it back to "all", I am able to create new index.

4

BTW: There is no any hint in Elasticsearch log why index creating failed. Is there any place to find detailed reason for such error?