Why is my status red? 0 relocating_shards, 0 initializing_shards, and 0 number_of_pending_tasks

Elastic Stack Elasticsearch
1

curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'
{
"cluster_name" : "elasticsearch",
"status" : "red",
"timed_out" : false,
"number_of_nodes" : 2,
"number_of_data_nodes" : 1,
"active_primary_shards" : 710,
"active_shards" : 710,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 710,
"number_of_pending_tasks" : 0
}

What else can I look at to see why it's red?

2

Did your node die for some reason?
Do you have enough disk space?

3

It has died before but it's been a while.

Plenty of space: (it's all in /opt/elasticsearch)
df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg_fslsologserve-LogVol01 20G 7.1G 12G 39% / tmpfs 12G 0 12G 0% /dev/shm /dev/vda1 477M 111M 341M 25% /boot /dev/mapper/vg_fslsologserve-LogVol02 555G 137G 391G 26% /opt

4

It might be worth looking at your ES logs then to see what is happening.

5

I did find this, it must be the issue because there are no other errors.

org.elasticsearch.action.NoShardAvailableActionException: [logstash-2015.06.15][4] null1

curl -XGET 'localhost:9200/logstash-2015.06.15/_search_shards?pretty=true' { "nodes" : { "10FqBhDnTyWVLcEx28tnqQ" : { "name" : "FSLSO-LOGSERVER", "transport_address" : "inet[/10.10.1.55:9301]" } }, "shards" : [ [ { "state" : "STARTED", "primary" : true, "node" : "10FqBhDnTyWVLcEx28tnqQ", "relocating_node" : null, "shard" : 0, "index" : "logstash-2015.06.15" } ], [ { "state" : "STARTED", "primary" : true, "node" : "10FqBhDnTyWVLcEx28tnqQ", "relocating_node" : null, "shard" : 1, "index" : "logstash-2015.06.15" } ], [ { "state" : "STARTED", "primary" : true, "node" : "10FqBhDnTyWVLcEx28tnqQ", "relocating_node" : null, "shard" : 2, "index" : "logstash-2015.06.15" } ], [ { "state" : "STARTED", "primary" : true, "node" : "10FqBhDnTyWVLcEx28tnqQ", "relocating_node" : null, "shard" : 3, "index" : "logstash-2015.06.15" } ], [ { "state" : "STARTED", "primary" : true, "node" : "10FqBhDnTyWVLcEx28tnqQ", "relocating_node" : null, "shard" : 4, "index" : "logstash-2015.06.15" } ] ] }

6

No, that is not the issue.

What does the output from _cat/indices show?