How to find which shard got corrupt


(Hanish Bansal) #1

Hi

Is there any way to find out which shard got corrupt.

Using elasticsearch cluster state we can determine which shard is
unassigned. But shards may be in unassigned state when any node is down or
any shard got corrupted.

I am looking for java api which can be used to find which shard got corrupt.

--
Thanks & Regards
Hanish Bansal

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jason Wee) #2

shard got corrupt? you mean index corrupt? if so, maybe you want to look
into index.shard.check_on_startup

/Jason

On Thu, Aug 29, 2013 at 4:21 PM, Hanish Bansal <
hanish.bansal.agarwal@gmail.com> wrote:

Hi

Is there any way to find out which shard got corrupt.

Using elasticsearch cluster state we can determine which shard is
unassigned. But shards may be in unassigned state when any node is down or
any shard got corrupted.

I am looking for java api which can be used to find which shard got
corrupt.

--
Thanks & Regards
Hanish Bansal

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Hanish Bansal) #3

Thanks for response Jason

Yes, i am looking for way to find corrupted index.
I set index.shard.check_on_startup : true but it is giving same behaviour.

I was asking only one case but now i would like to know:

  1. I have seen that ElasticSearch cluster state is GREEN and all shards are
    available even after index's one shard's data got corrupt. When i restart
    my cluster then its status change to RED and that shard goes to UNASSIGNED
    state. With and without setting index.shard.check_on_startup : true i am
    getting same behaviour. How to find which index got corrupt without any
    downtime
    ?
  2. Shards goes to UNASSIGNED state when they are not available. This may be
    due to (a) shard got corrupt (b) ES node is down. Then how to determine
    that shard's data got corrupt?

On Fri, Aug 30, 2013 at 9:18 AM, Jason Wee peichieh@gmail.com wrote:

shard got corrupt? you mean index corrupt? if so, maybe you want to look
into index.shard.check_on_startup

/Jason

On Thu, Aug 29, 2013 at 4:21 PM, Hanish Bansal <
hanish.bansal.agarwal@gmail.com> wrote:

Hi

Is there any way to find out which shard got corrupt.

Using elasticsearch cluster state we can determine which shard is
unassigned. But shards may be in unassigned state when any node is down or
any shard got corrupted.

I am looking for java api which can be used to find which shard got
corrupt.

--
Thanks & Regards
Hanish Bansal

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Thanks & Regards
Hanish Bansal

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jason Wee) #4

When you set the config to true, it should check the index during startup.
You can turn on necessary logs on java classes to see the process... but if
I am not mistaken, you can also set the config to fix so that if it
encounter index corruption, it will fix the corrupted index. If you have
replication of your shard, you should have able to bring down one node to
do maintenance , in your case, find which index is corrupt. there is a
lucene class that does index check,
http://java.dzone.com/news/lucene-and-solrs-checkindex . Maybe someone that
has experience in elasticsearch can comment / advice more on the index
check in es?

When a node goes down, the cluster should been able to reallocate the
unassigned shard to another node?

Jason

On Fri, Aug 30, 2013 at 9:16 PM, Hanish Bansal <
hanish.bansal.agarwal@gmail.com> wrote:

Thanks for response Jason

Yes, i am looking for way to find corrupted index.
I set index.shard.check_on_startup : true but it is giving same behaviour.

I was asking only one case but now i would like to know:

  1. I have seen that ElasticSearch cluster state is GREEN and all shards
    are available even after index's one shard's data got corrupt. When i
    restart my cluster then its status change to RED and that shard goes to
    UNASSIGNED state. With and without setting index.shard.check_on_startup :
    true i am getting same behaviour. How to find which index got corrupt without
    any downtime
    ?
  2. Shards goes to UNASSIGNED state when they are not available. This may
    be due to (a) shard got corrupt (b) ES node is down. Then how to determine
    that shard's data got corrupt?

On Fri, Aug 30, 2013 at 9:18 AM, Jason Wee peichieh@gmail.com wrote:

shard got corrupt? you mean index corrupt? if so, maybe you want to look
into index.shard.check_on_startup

/Jason

On Thu, Aug 29, 2013 at 4:21 PM, Hanish Bansal <
hanish.bansal.agarwal@gmail.com> wrote:

Hi

Is there any way to find out which shard got corrupt.

Using elasticsearch cluster state we can determine which shard is
unassigned. But shards may be in unassigned state when any node is down or
any shard got corrupted.

I am looking for java api which can be used to find which shard got
corrupt.

--
Thanks & Regards
Hanish Bansal

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Thanks & Regards
Hanish Bansal

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #5

The shards are numbered, in the log messages you can find out what shard
number is corrupt. In the cluster state info, you can lookup the node of
this shard number. On the node, you can find the shard data in the data
path.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #6