Not able to index data if one ES goes down in cluster

Hi All,

I am running 3 nodes ElasticSearch cluster. I have 2 shards in one index.

ES nodes details are as:

Node 0 --> serve shard0 of index livemass_ipdr_0
Node 1 --> serve shard1 of index livemass_ipdr_0
Node 2 --> No shard

where livemass_ipdr_0 is index name.

I am indexing continuous stream of records, if my node-0 goes down
(crashed) then i am getting below exception:
org.elasticsearch.action.NoShardAvailableActionException:
[livemass_ipdr_0][0] No shard available for
[[livemass_ipdr_0][livemass_ipdr][a84f412d873040c4804898b642b14b95]:
routing [null]

I am not able to index data if any node goes down which has shard.

How can i resolve this problem?

--
Thanks & Regards
Hanish Bansal

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hello!

This is expected behavior - when losing one primary shard (and you have no replicas) your index is not complete and the cluster goes to red health state. This means that indexing request will not be processed. You can for example add replicas for your index, which will allow some of your nodes to crash, but the cluster will still be operational.

For example if you would add another node to your cluster and have 1 replica, you would have something like this:

Node 0 - primary shard 0

Node 1 - primary shard 1

Node 2 - replica of primary shard 0

Node 3 - replica of primary shard 1

Now if your Node 0 would crash, the Node 2 (actually the replica shard) would take over the role of the primary shard and you would still be able to index data.

--

Regards,

Rafał Kuć

Sematext :: http://sematext.com/ :: Solr - Lucene - ElasticSearch

Hi All,

I am running 3 nodes ElasticSearch cluster. I have 2 shards in one index.

ES nodes details are as:

Node 0 --> serve shard0 of index livemass_ipdr_0

Node 1 --> serve shard1 of index livemass_ipdr_0

Node 2 --> No shard

where livemass_ipdr_0 is index name.

I am indexing continuous stream of records, if my node-0 goes down (crashed) then i am getting below exception:

org.elasticsearch.action.NoShardAvailableActionException: [livemass_ipdr_0][0] No shard available for [[livemass_ipdr_0][livemass_ipdr][a84f412d873040c4804898b642b14b95]: routing [null]

I am not able to index data if any node goes down which has shard.

How can i resolve this problem?

--

Thanks & Regards

Hanish Bansal

--

You received this message because you are subscribed to the Google Groups "elasticsearch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

Hi Rafal,

Thanks for quick response !!

I want to know one more thing: Is ES perform same behaviour in case of
searching the data ?

I mean if one shard not available(in case i have no replicas), can i search
the data on that index or not?

On Wed, Aug 14, 2013 at 6:37 PM, Rafał Kuć r.kuc@solr.pl wrote:

Hello!

This is expected behavior - when losing one primary shard (and you have no
replicas) your index is not complete and the cluster goes to red health
state. This means that indexing request will not be processed. You can for
example add replicas for your index, which will allow some of your nodes to
crash, but the cluster will still be operational.

For example if you would add another node to your cluster and have 1
replica, you would have something like this:

Node 0 - primary shard 0
Node 1 - primary shard 1
Node 2 - replica of primary shard 0
Node 3 - replica of primary shard 1

Now if your Node 0 would crash, the Node 2 (actually the replica shard)
would take over the role of the primary shard and you would still be able
to index data.

*--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - Elasticsearch

Hi All,

I am running 3 nodes Elasticsearch cluster. I have 2 shards in one index.

ES nodes details are as:

Node 0 --> serve shard0 of index livemass_ipdr_0
Node 1 --> serve shard1 of index livemass_ipdr_0
Node 2 --> No shard
where livemass_ipdr_0 is index name.

I am indexing continuous stream of records, if my node-0 goes down
(crashed) then i am getting below exception:
org.elasticsearch.action.NoShardAvailableActionException:
[livemass_ipdr_0][0] No shard available for
[[livemass_ipdr_0][livemass_ipdr][a84f412d873040c4804898b642b14b95]:
routing [null]

I am not able to index data if any node goes down which has shard.

How can i resolve this problem?

--
*Thanks & Regards
Hanish Bansal
*

You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Thanks & Regards
Hanish Bansal

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hello!

As for the search - you should still be able to search, however some portion of your data (in your case about 50% of it with the default routing) will be unavailable. When querying, ElasticSearch will tell you how many shards were queried and how many of them were successful or failed, for example:

{

"took" : 115,

"timed_out" : false,

"_shards" : {

"total" : 5,


"successful" : 4,


"failed" : 0

},

"hits" : {

...

}

}

--

Regards,

Rafał Kuć

Sematext :: http://sematext.com/ :: Solr - Lucene - ElasticSearch

Hi Rafal,

Thanks for quick response !!

I want to know one more thing: Is ES perform same behaviour in case of searching the data ?

I mean if one shard not available(in case i have no replicas), can i search the data on that index or not?

On Wed, Aug 14, 2013 at 6:37 PM, Rafał Kuć <r.kuc@solr.pl> wrote:

Hello!

This is expected behavior - when losing one primary shard (and you have no replicas) your index is not complete and the cluster goes to red health state. This means that indexing request will not be processed. You can for example add replicas for your index, which will allow some of your nodes to crash, but the cluster will still be operational.

For example if you would add another node to your cluster and have 1 replica, you would have something like this:

Node 0 - primary shard 0

Node 1 - primary shard 1

Node 2 - replica of primary shard 0

Node 3 - replica of primary shard 1

Now if your Node 0 would crash, the Node 2 (actually the replica shard) would take over the role of the primary shard and you would still be able to index data.

--

Regards,

Rafał Kuć

Sematext :: http://sematext.com/ :: Solr - Lucene - ElasticSearch

Hi All,

I am running 3 nodes ElasticSearch cluster. I have 2 shards in one index.

ES nodes details are as:

Node 0 --> serve shard0 of index livemass_ipdr_0

Node 1 --> serve shard1 of index livemass_ipdr_0

Node 2 --> No shard

where livemass_ipdr_0 is index name.

I am indexing continuous stream of records, if my node-0 goes down (crashed) then i am getting below exception:

org.elasticsearch.action.NoShardAvailableActionException: [livemass_ipdr_0][0] No shard available for [[livemass_ipdr_0][livemass_ipdr][a84f412d873040c4804898b642b14b95]: routing [null]

I am not able to index data if any node goes down which has shard.

How can i resolve this problem?

--

Thanks & Regards

Hanish Bansal

--

You received this message because you are subscribed to the Google Groups "elasticsearch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

You received this message because you are subscribed to the Google Groups "elasticsearch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

--

Thanks & Regards

Hanish Bansal

--

You received this message because you are subscribed to the Google Groups "elasticsearch" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.

Thanks Rafal :slight_smile:

On Wed, Aug 14, 2013 at 7:03 PM, Rafał Kuć r.kuc@solr.pl wrote:

Hello!

As for the search - you should still be able to search, however some
portion of your data (in your case about 50% of it with the default
routing) will be unavailable. When querying, Elasticsearch will tell you
how many shards were queried and how many of them were successful or
failed, for example:

{
"took" : 115,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 4,
"failed" : 0
},
"hits" : {
...

}
}

--
Regards,
Rafał Kuć
Sematext :: *
http://sematext.com/
:: Solr - Lucene - Elasticsearch

Hi Rafal,

Thanks for quick response !!

I want to know one more thing: Is ES perform same behaviour in case of
searching the data ?

I mean if one shard not available(in case i have no replicas), can i
search the data on that index or not?

On Wed, Aug 14, 2013 at 6:37 PM, Rafał Kuć r.kuc@solr.pl wrote:
Hello!

This is expected behavior - when losing one primary shard (and you have no
replicas) your index is not complete and the cluster goes to red health
state. This means that indexing request will not be processed. You can for
example add replicas for your index, which will allow some of your nodes to
crash, but the cluster will still be operational.

For example if you would add another node to your cluster and have 1
replica, you would have something like this:

Node 0 - primary shard 0
Node 1 - primary shard 1
Node 2 - replica of primary shard 0
Node 3 - replica of primary shard 1

Now if your Node 0 would crash, the Node 2 (actually the replica shard)
would take over the role of the primary shard and you would still be able
to index data.

*--
Regards,
Rafał Kuć
Sematext :: http://sematext.com/ :: Solr - Lucene - Elasticsearch

Hi All,

I am running 3 nodes Elasticsearch cluster. I have 2 shards in one index.

ES nodes details are as:

Node 0 --> serve shard0 of index livemass_ipdr_0
Node 1 --> serve shard1 of index livemass_ipdr_0
Node 2 --> No shard
where livemass_ipdr_0 is index name.

I am indexing continuous stream of records, if my node-0 goes down
(crashed) then i am getting below exception:
org.elasticsearch.action.NoShardAvailableActionException:
[livemass_ipdr_0][0] No shard available for
[[livemass_ipdr_0][livemass_ipdr][a84f412d873040c4804898b642b14b95]:
routing [null]

I am not able to index data if any node goes down which has shard.

How can i resolve this problem?

--
*Thanks & Regards
Hanish Bansal
*--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
*Thanks & Regards
Hanish Bansal
*--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Thanks & Regards
Hanish Bansal

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.