Sending failed shard


(David Chamard) #1

Hi,

I am currently having an issue that causes data loss with failed shards. At
first sight it seems related to invalid mappings/parsing.
Is that possible that a shard will fail to start due to that kind of issues?

There is the Gist:

[2014-04-11 18:04:22,166][WARN ][cluster.action.shard ]
[elastic09.localdomain] [logstash-2014.04.11][3] sending failed shard for
[logstash-2014.04.11][3], node[GrR8qRaeQLOKf3jhxvJJzw], [R],
s[INITIALIZING], indexUUID [xMDsVqv7TwmQAxMq3buWPw], reason [Failed to
start shard, message [RecoveryFailedException[[logstash-2014.04.11][3]:
Recovery failed from
[elastic08.localdomain][-hBCZQqVQ92wn0qTFZXCBQ][inet[/10.0.241.154:9300]]{rack_id=us-east-1b,
aws_availability_zone=us-east-1b, max_local_storage_nodes=1, master=false}
into
[elastic09.localdomain][GrR8qRaeQLOKf3jhxvJJzw][inet[/10.0.241.183:9300]]{rack_id=us-east-1c,
aws_availability_zone=us-east-1c, max_local_storage_nodes=1,
master=false}]; nested:
RemoteTransportException[[elastic08.localdomain][inet[/10.0.241.154:9300]][index/shard/recovery/startRecovery]];
nested: RecoveryEngineException[[logstash-2014.04.11][3] Phase[2] Execution
failed]; nested:
RemoteTransportException[[elastic09.localdomain][inet[/10.0.241.183:9300]][index/shard/recovery/translogOps]];
nested: MapperParsingException[failed to parse [result]]; nested:
ElasticSearchIllegalArgumentException[unknown property [total_rows]]; ]]

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/08ecbd3a-502f-4576-a9d8-8965ebf7d72f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Alexander Reelsen) #2

Hey,

has there a stacktrace been logged below this message? Also the
elasticsearch version would be interesting..

--Alex

On Fri, Apr 11, 2014 at 8:14 PM, David Chamard chammmmm@gmail.com wrote:

Hi,

I am currently having an issue that causes data loss with failed shards.
At first sight it seems related to invalid mappings/parsing.
Is that possible that a shard will fail to start due to that kind of
issues?

There is the Gist:

https://gist.github.com/anonymous/d8a692d28a1cf14f961b

[2014-04-11 18:04:22,166][WARN ][cluster.action.shard ]
[elastic09.localdomain] [logstash-2014.04.11][3] sending failed shard for
[logstash-2014.04.11][3], node[GrR8qRaeQLOKf3jhxvJJzw], [R],
s[INITIALIZING], indexUUID [xMDsVqv7TwmQAxMq3buWPw], reason [Failed to
start shard, message [RecoveryFailedException[[logstash-2014.04.11][3]:
Recovery failed from
[elastic08.localdomain][-hBCZQqVQ92wn0qTFZXCBQ][inet[/10.0.241.154:9300]]{rack_id=us-east-1b,
aws_availability_zone=us-east-1b, max_local_storage_nodes=1, master=false}
into [elastic09.localdomain][GrR8qRaeQLOKf3jhxvJJzw][inet[/10.0.241.183:9300]]{rack_id=us-east-1c,
aws_availability_zone=us-east-1c, max_local_storage_nodes=1,
master=false}]; nested:
RemoteTransportException[[elastic08.localdomain][inet[/10.0.241.154:9300]][index/shard/recovery/startRecovery]];
nested: RecoveryEngineException[[logstash-2014.04.11][3] Phase[2] Execution
failed]; nested:
RemoteTransportException[[elastic09.localdomain][inet[/10.0.241.183:9300]][index/shard/recovery/translogOps]];
nested: MapperParsingException[failed to parse [result]]; nested:
ElasticSearchIllegalArgumentException[unknown property [total_rows]]; ]]

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/08ecbd3a-502f-4576-a9d8-8965ebf7d72f%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/08ecbd3a-502f-4576-a9d8-8965ebf7d72f%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-JWy2RJOa--pHb6JnXycEK1Pkgb%2B%3D-q8CaOu2P5aVmjg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #3