Cluster status: red

Elastic Stack Elasticsearch
1

Using Elasticsearch for Logstash. Everything was working. Came back after
lunch, no results.

Started digging further, found this:

curl -XGET 'http://127.0.0.1:9200/_cluster/health?pretty=true'
{
"cluster_name" : "elasticsearch",
"status" : "red",
"timed_out" : false,
"number_of_nodes" : 2,
"number_of_data_nodes" : 1,
"active_primary_shards" : 0,
"active_shards" : 0,
"relocating_shards" : 0,
"initializing_shards" : 4,
"unassigned_shards" : 76
}

Various threads have shown me, this is bad.

Best that I can tell from the logs, this is when it happened
https://gist.github.com/esacteksab/6033983

Anyway to get those unassigned_shards back?

Logstash/Elasticsearch exist on the same server, "defaults" otherwise.
Everything has been working for over a week...until a few hours ago.

Thanks,
Barry

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Are you using different es versions in one cluster? Could this cause an error?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Originally I stood up on 0.90.2. And had this problem. It was then
mentioned to go to 0.20.6. Problem still exists, regardless of the version.

I am now still on 0.20.6.

On Thu, Jul 18, 2013 at 4:57 PM, cthoma thch0014@gmail.com wrote:

Are you using different es versions in one cluster? Could this cause an
error?

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/vnjNZRsdxhY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

More information for the sake of information:

curl -XGET 'http://127.0.0.1:9200/_cluster/nodes/_local?pretty=true'
{
"ok" : true,
"cluster_name" : "elasticsearch",
"nodes" : {
"3Vne3-D2Q-S0Fu8Q4aMpmA" : {
"name" : "Kate Neville",
"transport_address" :
"inet[ip-10-245-10-23.us-west-2.compute.internal/10.245.10.23:9300]",
"hostname" : "logger00.mine.net",
"version" : "0.20.6",
"http_address" : "inet[/10.245.10.23:9200]"
}
}
}

On Thursday, July 18, 2013 4:37:31 PM UTC-7, Barry Morrison wrote:

Using Elasticsearch for Logstash. Everything was working. Came back after
lunch, no results.

Started digging further, found this:

curl -XGET 'http://127.0.0.1:9200/_cluster/health?pretty=true'
{
"cluster_name" : "elasticsearch",
"status" : "red",
"timed_out" : false,
"number_of_nodes" : 2,
"number_of_data_nodes" : 1,
"active_primary_shards" : 0,
"active_shards" : 0,
"relocating_shards" : 0,
"initializing_shards" : 4,
"unassigned_shards" : 76
}

Various threads have shown me, this is bad.

Best that I can tell from the logs, this is when it happened
https://gist.github.com/esacteksab/6033983

Anyway to get those unassigned_shards back?

Logstash/Elasticsearch exist on the same server, "defaults" otherwise.
Everything has been working for over a week...until a few hours ago.

Thanks,
Barry

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

5

Hey,

minor question: did you try to use the same data files after downgrading
from 0.90 to 0.20? (they might not be compatible in that direction)

--Alex

On Fri, Jul 19, 2013 at 2:05 AM, Barry Morrison barry@versal.com wrote:

More information for the sake of information:

curl -XGET 'http://127.0.0.1:9200/_cluster/nodes/_local?pretty=true'
{
"ok" : true,
"cluster_name" : "elasticsearch",
"nodes" : {
"3Vne3-D2Q-S0Fu8Q4aMpmA" : {
"name" : "Kate Neville",
"transport_address" :
"inet[ip-10-245-10-23.us-west-2.compute.internal/10.245.10.23:9300]",
"hostname" : "logger00.mine.net",
"version" : "0.20.6",
"http_address" : "inet[/10.245.10.23:9200]"
}
}
}

On Thursday, July 18, 2013 4:37:31 PM UTC-7, Barry Morrison wrote:

Using Elasticsearch for Logstash. Everything was working. Came back after
lunch, no results.

Started digging further, found this:

curl -XGET 'http://127.0.0.1:9200/_**cluster/health?pretty=truehttp://127.0.0.1:9200/_cluster/health?pretty=true
'
{
"cluster_name" : "elasticsearch",
"status" : "red",
"timed_out" : false,
"number_of_nodes" : 2,
"number_of_data_nodes" : 1,
"active_primary_shards" : 0,
"active_shards" : 0,
"relocating_shards" : 0,
"initializing_shards" : 4,
"unassigned_shards" : 76
}

Various threads have shown me, this is bad.

Best that I can tell from the logs, this is when it happened
https://gist.github.**com/esacteksab/6033983https://gist.github.com/esacteksab/6033983

Anyway to get those unassigned_shards back?

Logstash/Elasticsearch exist on the same server, "defaults" otherwise.
Everything has been working for over a week...until a few hours ago.

Thanks,
Barry

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

6

Alex,
The problem started/existed with 0.90.2.

Logstash folks recommended 0.20.6. Problem still exists.

So while the data may be incompatible, it is still broken, regardless of
the version.

On Thu, Jul 18, 2013 at 11:32 PM, Alexander Reelsen alr@spinscale.dewrote:

Hey,

minor question: did you try to use the same data files after downgrading
from 0.90 to 0.20? (they might not be compatible in that direction)

--Alex

On Fri, Jul 19, 2013 at 2:05 AM, Barry Morrison barry@versal.com wrote:

More information for the sake of information:

curl -XGET 'http://127.0.0.1:9200/_cluster/nodes/_local?pretty=true'
{
"ok" : true,
"cluster_name" : "elasticsearch",
"nodes" : {
"3Vne3-D2Q-S0Fu8Q4aMpmA" : {
"name" : "Kate Neville",
"transport_address" :
"inet[ip-10-245-10-23.us-west-2.compute.internal/10.245.10.23:9300]",
"hostname" : "logger00.mine.net",
"version" : "0.20.6",
"http_address" : "inet[/10.245.10.23:9200]"
}
}
}

On Thursday, July 18, 2013 4:37:31 PM UTC-7, Barry Morrison wrote:

Using Elasticsearch for Logstash. Everything was working. Came back
after lunch, no results.

Started digging further, found this:

curl -XGET 'http://127.0.0.1:9200/_**cluster/health?pretty=truehttp://127.0.0.1:9200/_cluster/health?pretty=true
'
{
"cluster_name" : "elasticsearch",
"status" : "red",
"timed_out" : false,
"number_of_nodes" : 2,
"number_of_data_nodes" : 1,
"active_primary_shards" : 0,
"active_shards" : 0,
"relocating_shards" : 0,
"initializing_shards" : 4,
"unassigned_shards" : 76
}

Various threads have shown me, this is bad.

Best that I can tell from the logs, this is when it happened
https://gist.github.**com/esacteksab/6033983https://gist.github.com/esacteksab/6033983

Anyway to get those unassigned_shards back?

Logstash/Elasticsearch exist on the same server, "defaults" otherwise.
Everything has been working for over a week...until a few hours ago.

Thanks,
Barry

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/vnjNZRsdxhY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.