MapperParsingException on data that should not be parsed but caught by my plugin

Laurent_T · June 4, 2014, 4:30pm

Hi,

I've just activated DEBUG mode on my ES logs and i'm seeing this kind of
exception:

[2014-06-04 15:50:03,539][DEBUG][action.index ] [Supercharger] [
myplugin][0], node[drCfkhlURn2Yz_SsM6bD3w], [P], s[STARTED]: Failed to
execute [index {[myplugin][client1][69Pb9C_kT6CF6jdH06WbJw],
source[{"value":"Cr�py-en-valois"}]}]
org.elasticsearch.index.mapper.MapperParsingException: failed to parse
[value]
at
org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:396)
at
org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:599)
at
org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:467)
at
org.elasticsearch.index.mapper.object.ObjectMapper.serializeObject(ObjectMapper.java:515)
at
org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:457)
at
org.elasticsearch.index.mapper.object.ObjectMapper.serializeObject(ObjectMapper.java:515)
at
org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:457)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:507)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:451)
at
org.elasticsearch.index.shard.service.InternalIndexShard.prepareCreate(InternalIndexShard.java:308)
at
org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:211)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:521)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:419)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.common.jackson.core.JsonParseException:
Invalid UTF-8 middle byte 0x70
at [Source: [B@2d1b0cf9; line: 1, column: 11]
at
org.elasticsearch.common.jackson.core.JsonParser._constructError(JsonParser.java:1369)
at
org.elasticsearch.common.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:599)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._reportInvalidOther(UTF8StreamJsonParser.java:3004)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._reportInvalidOther(UTF8StreamJsonParser.java:3011)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._decodeUtf8_3fast(UTF8StreamJsonParser.java:2833)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:2135)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._finishString(UTF8StreamJsonParser.java:2084)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:270)
at
org.elasticsearch.common.xcontent.json.JsonXContentParser.text(JsonXContentParser.java:85)
at
org.elasticsearch.common.xcontent.support.AbstractXContentParser.textOrNull(AbstractXContentParser.java:107)
at
org.elasticsearch.index.mapper.core.StringFieldMapper.parseCreateField(StringFieldMapper.java:285)
at
org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:385)
... 15 more

I'm wondering why this is happening and why this error is shown only in
DEBUG mode.
This request should actually be targeting my plugin that would JSON-decode
it using Google Gson.

Is ES doing anything else before forwarding the request to the plugin ?
We're using version 0.90.3 of ES.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f6afec0d-24f5-4438-b5d4-3e2d6b87c5c6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

spinscale · June 16, 2014, 12:18pm

Hey,

first you should really upgrade ES, 0.90.3 is not the freshest one and
there have been many improvements.
Second, logging this as DEBUG makes a lot of sense, as otherwise you
logfiles might explode, just because you are indexing a lot of bogus
documents. You source of truth should not be the logfile, but the error
message, which should have been returned upon indexing this JSON document.

It seems, as if the data you sent is not valid UTF-8, but some other
encoding...

--Alex

On Wed, Jun 4, 2014 at 6:30 PM, Laurent T. lau.thoulon@gmail.com wrote:

Hi,

I've just activated DEBUG mode on my ES logs and i'm seeing this kind of
exception:

[2014-06-04 15:50:03,539][DEBUG][action.index ] [Supercharger]
[myplugin][0], node[drCfkhlURn2Yz_SsM6bD3w], [P], s[STARTED]: Failed to
execute [index {[myplugin][client1][69Pb9C_kT6CF6jdH06WbJw],
source[{"value":"Cr�py-en-valois"}]}]
org.elasticsearch.index.mapper.MapperParsingException: failed to parse
[value]
at
org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:396)
at
org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:599)
at
org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:467)
at
org.elasticsearch.index.mapper.object.ObjectMapper.serializeObject(ObjectMapper.java:515)
at
org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:457)
at
org.elasticsearch.index.mapper.object.ObjectMapper.serializeObject(ObjectMapper.java:515)
at
org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:457)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:507)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:451)
at
org.elasticsearch.index.shard.service.InternalIndexShard.prepareCreate(InternalIndexShard.java:308)
at
org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:211)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:521)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:419)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.common.jackson.core.JsonParseException:
Invalid UTF-8 middle byte 0x70
at [Source: [B@2d1b0cf9; line: 1, column: 11]
at
org.elasticsearch.common.jackson.core.JsonParser._constructError(JsonParser.java:1369)
at
org.elasticsearch.common.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:599)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._reportInvalidOther(UTF8StreamJsonParser.java:3004)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._reportInvalidOther(UTF8StreamJsonParser.java:3011)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._decodeUtf8_3fast(UTF8StreamJsonParser.java:2833)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:2135)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser._finishString(UTF8StreamJsonParser.java:2084)
at
org.elasticsearch.common.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:270)
at
org.elasticsearch.common.xcontent.json.JsonXContentParser.text(JsonXContentParser.java:85)
at
org.elasticsearch.common.xcontent.support.AbstractXContentParser.textOrNull(AbstractXContentParser.java:107)
at
org.elasticsearch.index.mapper.core.StringFieldMapper.parseCreateField(StringFieldMapper.java:285)
at
org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:385)
... 15 more

I'm wondering why this is happening and why this error is shown only in
DEBUG mode.
This request should actually be targeting my plugin that would JSON-decode
it using Google Gson.

Is ES doing anything else before forwarding the request to the plugin ?
We're using version 0.90.3 of ES.

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f6afec0d-24f5-4438-b5d4-3e2d6b87c5c6%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/f6afec0d-24f5-4438-b5d4-3e2d6b87c5c6%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM8PFaSff35UED%3DVj0yaqMWK1Fmd2NWg_tzb2Tt%2BAiVPyA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.