Hi all,
We're having a MapperParsingException problem with some field values when
we get when we use the JSON Filter for Logstash to explode out a JSON
document to Elasticsearch fields.
In 99.9% of cases, certain of these fields are either blank, or contain
dates in the format of yyyy-mm-dd. This allows ES to dynamically map this
field to type dateOptionalTime.
However, we occasionally see non-standard date formats in these fields,
which our main service can handle fine, but which throws a
MapperParsingException in Elasticsearch - such are here:
[2015-02-06 10:46:50,679][WARN ][cluster.action.shard ] [logging-
production-elasticsearch-ip-xxx-xxx-xxx-148] [logstash-2015.02.06][2]
received shard failed for [logstash-2015.02.06][2], node[
GZpltBjAQUqGyp2B1SLz_g], [R], s[INITIALIZING], indexUUID [BEdTwj-
QRuOZB713YAQwvA], reason [Failed to start shard, message [
RecoveryFailedException[[logstash-2015.02.06][2]: Recovery failed from [
logging-production-elasticsearch-ip-xxx-xxx-xxx-82][IALW-92RReiLffQjSL3I-g][
logging-production-elasticsearch-ip-xxx-xxx-xxx-82][inet[ip-xxx-xxx-xxx-
82.ec2.internal/xxx.xxx.xxx.82:9300]]{max_local_storage_nodes=1,
aws_availability_zone=us-east-1e, aws_az=us-east-1e} into [logging-
production-elasticsearch-ip-xxx-xxx-xxx-148][GZpltBjAQUqGyp2B1SLz_g][logging
-production-elasticsearch-ip-xxx-xxx-xxx-148][inet[ip-xxx.xxx.xxx.
148.ec2.internal/xxx.xxx.xxx.148:9300]]{max_local_storage_nodes=1,
aws_availability_zone=us-east-1c, aws_az=us-east-1c}]; nested:
RemoteTransportException[[logging-production-elasticsearch-ip-xxx-xxx-xxx-82
][inet[/xxx.xxx.xxx.82:9300]][internal:index/shard/recovery/start_recovery
]]; nested: RecoveryEngineException[[logstash-2015.02.06][2] Phase[2]
Execution failed]; nested: RemoteTransportException[[logging-production-
elasticsearch-ip-xxx-xxx-xxx-148][inet[/xxx.xxx.xxx.148:9300]][internal:
index/shard/recovery/translog_ops]]; nested: MapperParsingException[failed
to parse [apiservice.logstash.@fields.parameters.start_time]]; nested:
MapperParsingException[failed to parse date field [Feb 5 2015 12:00 AM],
tried both date format [dateOptionalTime], and timestamp number with locale
[]]; nested: IllegalArgumentException[Invalid format: "Feb 5 2015 12:00 AM"
]; ]]
2015-02-06 10:46:53,685][WARN ][cluster.action.shard ] [logging-
production-elasticsearch-ip-xxx-xxx-xxx-148] [logstash-2015.02.06][2]
received shard failed for [logstash-2015.02.06][2], node[
GZpltBjAQUqGyp2B1SLz_g], [R], s[INITIALIZING], indexUUID [BEdTwj-
QRuOZB713YAQwvA], reason [master [logging-production-elasticsearch-ip-xxx-
xxx-xxx-148][GZpltBjAQUqGyp2B1SLz_g][logging-production-elasticsearch-ip-xxx
-xxx-xxx-148][inet[ip-xxx-xxx-xxx-148.ec2.internal/xxx.xxx.xxx.148:9300]]{
max_local_storage_nodes=1, aws_availability_zone=us-east-1c, aws_az=us-east-
1c} marked shard as initializing, but shard is marked as failed, resend
shard failure]
Our planned solution was to create a template for Logstash indices that
will set these fields to string. But as the field above isn't the only
culprit, and more may be added overtime, it makes more sense to create a
template to map all fields under apiservice.logstash.@fields.parameters.*
to be string. (We never need to query on user entered data, but it's great
to have logged for debugging)
Is it possible to do this with a template? I could not find a way to do
this via the template documentation on the ES site.
Any guidance would be great!
Thanks,
-Paul
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6ca4030f-b6bb-4907-b2fc-e3166fa2a6af%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.