Org.elasticsearch.index.mapper.MapperParsingException: failed to parse -- NEED HELP

Hi,

Hello. I have read about similar problems online but haven't really figured
out what the solution is, and was hoping somebody can point me in the right
direction.

I'm using ELK.

ES 1.5.0
Logstash 1.5.0rc2
Kibana 4.0.1

I have all type of application logs that are written out as JSON, but the
JSON can be nested, and there are values that can sometimes be null. If I
delete all my indices, and restart everything from scratch (restart
elasticsearch, kibana and logstash), everything works fine. Every 2 or 3
weeks, I start seeing errors such as these:

[2015-04-21 21:15:22,197][DEBUG][action.bulk ]
[elasticsearch09] [logstash-2015.04.21][20] failed to execute bulk item
(index) index {[logstash-2015.04.21][exchange][AUzd1drI5RY1eWdGgIZm],
source[{"log_type":"error_server_response","app_name":"Adap.TV Advertising
iOS","app_id":"538e2980a1df5de7790000e7","pub_app_id":"653967729","streaming_url":"http://u-ads.adap.tv/a/h/eaLAQ7VgTRxUq8XV5RJZpiQfAqV5UMkXExLyYUmEPHM=?cb=2015-04-21T05%3A16%3A28%2B00%3A00&pet=preroll&pageUrl=apps%3A%2F%2FCharades!%20Guess%20Words%20and%20Draw%20or%20Doodle%20Something%20Taboo%20Tilt%20Your%20Heads%20Jump%20Up%20with%20Friends%20Free&a.ip=76.165.97.2&a.aid=15eaddd2-52fe-401f-b33c-333f5f78e01f&a.idfa=15eaddd2-52fe-401f-b33c-333f5f78e01f&a.ua=Mozilla%2F5.0%20(iPhone%3B%20CPU%20iPhone%20OS%207_1_1%20like%20Mac%20OS%20X)%20AppleWebKit%2F537.51.2%20(KHTML%2C%20like%20Gecko)%20Mobile%2F11D201&eov=eov","error":"Cannot
have empty
VAST","host":"ip-10-233-53-75","level":"info","message":"","timestamp":"2015-04-21
05:16:29.021","@version":"1","@timestamp":"2015-04-21T21:15:22.087Z","type":"exchange"}]}
org.elasticsearch.index.mapper.MapperParsingException: object mapping for
[exchange] tried to parse field [error] as object, but got EOF, has a
concrete value been provided to it?
at
org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:495)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:544)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:493)
at
org.elasticsearch.index.shard.IndexShard.prepareCreate(IndexShard.java:438)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:432)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:149)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:515)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:422)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

AND

[2015-04-21 21:19:09,286][DEBUG][action.bulk ]
[elasticsearch03] [logstash-2015.04.21][19] failed to execute bulk item
(index) index {[logstash-2015.04.21][requestAds][AUzd2VGHPBf0vCHmQV4j],
source[{"country":"BR","region":"18","city":"Curitiba","latitude":null,"longitude":null,"device_language":"pt","browser_user_agent":"VungleDroid/3.3.0","is_sd_card_available":1,"device_make":"motorola","device_model":"XT1033","device_height":1184,"device_width":720,"os_version":"5.0.2","platform":"android","sound_enabled":false,"volume":0,"device_id":"5ec495a3-80cb-4682-a0a9-c66c2ca122ea","ifa":"5ec495a3-80cb-4682-a0a9-c66c2ca122ea","isu":"5ec495a3-80cb-4682-a0a9-c66c2ca122ea","user_age":null,"user_gender":null,"ip_address":"189.123.219.242","connection":"wifi","network_operator":"TIM","pub_app_id":"507686ae771615941001aca5","pub_app_bundle_id":"com.kiloo.subwaysurf","ad_app_id":null,"campaign_id":null,"creative_id":null,"event_id":null,"sleep":-1,"strategy":null,"expiry":null,"post_bundle":null,"video_url":null,"show_close":null,"show_close_incentivized":null,"video_height":null,"video_width":null,"call_to_action_url":null,"call_to_action_destination":null,"countdown":null,"delay":null,"error":"Cached
ad is
better","shouldStream":false,"message":"","is_test":false,"host":"ip-10-155-170-179","level":"info","timestamp":"2015-04-21
21:18:21.280","@version":"1","@timestamp":"2015-04-21T21:19:07.996Z","type":"requestAds"}]}
org.elasticsearch.index.mapper.MapperParsingException: failed to parse
[error]
at
org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:410)
at
org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:706)
at
org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:497)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:544)
at
org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:493)
at
org.elasticsearch.index.shard.IndexShard.prepareCreate(IndexShard.java:438)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:432)
at
org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:149)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:515)
at
org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:422)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NumberFormatException: For input string: "Cached ad is
better"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:441)
at java.lang.Long.parseLong(Long.java:483)
at
org.elasticsearch.common.xcontent.support.AbstractXContentParser.longValue(AbstractXContentParser.java:145)
at
org.elasticsearch.index.mapper.core.LongFieldMapper.innerParseCreateField(LongFieldMapper.java:300)
at
org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:236)
at
org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:400)
... 12 more

I'm wondering if I have to build custom indicies? This didn't seem to be a
problem with my older versions of Logstash and Elasticsearch.

Thanks!

Tony

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/285fb32b-abf5-4d42-94f2-7ca58c75dcf7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

I think you mapping expects an object for the error field but you sent a string in it.

--
David Pilato - Developer | Evangelist
elastic.co
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr https://twitter.com/elasticsearchfr | @scrutmydocs https://twitter.com/scrutmydocs

Le 21 avr. 2015 à 23:23, Tony Chong tony.chong@vungle.com a écrit :

Hi,

Hello. I have read about similar problems online but haven't really figured out what the solution is, and was hoping somebody can point me in the right direction.

I'm using ELK.

ES 1.5.0
Logstash 1.5.0rc2
Kibana 4.0.1

I have all type of application logs that are written out as JSON, but the JSON can be nested, and there are values that can sometimes be null. If I delete all my indices, and restart everything from scratch (restart elasticsearch, kibana and logstash), everything works fine. Every 2 or 3 weeks, I start seeing errors such as these:

[2015-04-21 21:15:22,197][DEBUG][action.bulk ] [elasticsearch09] [logstash-2015.04.21][20] failed to execute bulk item (index) index {[logstash-2015.04.21][exchange][AUzd1drI5RY1eWdGgIZm], source[{"log_type":"error_server_response","app_name":"Adap.TV Advertising iOS","app_id":"538e2980a1df5de7790000e7","pub_app_id":"653967729","streaming_url":"http://u-ads.adap.tv/a/h/eaLAQ7VgTRxUq8XV5RJZpiQfAqV5UMkXExLyYUmEPHM=?cb=2015-04-21T05%3A16%3A28%2B00%3A00&pet=preroll&pageUrl=apps%3A%2F%2FCharades!%20Guess%20Words%20and%20Draw%20or%20Doodle%20Something%20Taboo%20Tilt%20Your%20Heads%20Jump%20Up%20with%20Friends%20Free&a.ip=76.165.97.2&a.aid=15eaddd2-52fe-401f-b33c-333f5f78e01f&a.idfa=15eaddd2-52fe-401f-b33c-333f5f78e01f&a.ua=Mozilla%2F5.0%20(iPhone%3B%20CPU%20iPhone%20OS%207_1_1%20like%20Mac%20OS%20X)%20AppleWebKit%2F537.51.2%20(KHTML%2C%20like%20Gecko)%20Mobile%2F11D201&eov=eov","error":"Cannot have empty VAST","host":"ip-10-233-53-75","level":"info","message":"","timestamp":"2015-04-21 05:16:29.021","@version":"1","@timestamp":"2015-04-21T21:15:22.087Z","type":"exchange"}]}
org.elasticsearch.index.mapper.MapperParsingException: object mapping for [exchange] tried to parse field [error] as object, but got EOF, has a concrete value been provided to it?
at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:495)
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:544)
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:493)
at org.elasticsearch.index.shard.IndexShard.prepareCreate(IndexShard.java:438)
at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:432)
at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:149)
at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:515)
at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:422)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

AND

[2015-04-21 21:19:09,286][DEBUG][action.bulk ] [elasticsearch03] [logstash-2015.04.21][19] failed to execute bulk item (index) index {[logstash-2015.04.21][requestAds][AUzd2VGHPBf0vCHmQV4j], source[{"country":"BR","region":"18","city":"Curitiba","latitude":null,"longitude":null,"device_language":"pt","browser_user_agent":"VungleDroid/3.3.0","is_sd_card_available":1,"device_make":"motorola","device_model":"XT1033","device_height":1184,"device_width":720,"os_version":"5.0.2","platform":"android","sound_enabled":false,"volume":0,"device_id":"5ec495a3-80cb-4682-a0a9-c66c2ca122ea","ifa":"5ec495a3-80cb-4682-a0a9-c66c2ca122ea","isu":"5ec495a3-80cb-4682-a0a9-c66c2ca122ea","user_age":null,"user_gender":null,"ip_address":"189.123.219.242","connection":"wifi","network_operator":"TIM","pub_app_id":"507686ae771615941001aca5","pub_app_bundle_id":"com.kiloo.subwaysurf","ad_app_id":null,"campaign_id":null,"creative_id":null,"event_id":null,"sleep":-1,"strategy":null,"expiry":null,"post_bundle":null,"video_url":null,"show_close":null,"show_close_incentivized":null,"video_height":null,"video_width":null,"call_to_action_url":null,"call_to_action_destination":null,"countdown":null,"delay":null,"error":"Cached ad is better","shouldStream":false,"message":"","is_test":false,"host":"ip-10-155-170-179","level":"info","timestamp":"2015-04-21 21:18:21.280","@version":"1","@timestamp":"2015-04-21T21:19:07.996Z","type":"requestAds"}]}
org.elasticsearch.index.mapper.MapperParsingException: failed to parse [error]
at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:410)
at org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:706)
at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:497)
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:544)
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:493)
at org.elasticsearch.index.shard.IndexShard.prepareCreate(IndexShard.java:438)
at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:432)
at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:149)
at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:515)
at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:422)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NumberFormatException: For input string: "Cached ad is better"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:441)
at java.lang.Long.parseLong(Long.java:483)
at org.elasticsearch.common.xcontent.support.AbstractXContentParser.longValue(AbstractXContentParser.java:145)
at org.elasticsearch.index.mapper.core.LongFieldMapper.innerParseCreateField(LongFieldMapper.java:300)
at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:236)
at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:400)
... 12 more

I'm wondering if I have to build custom indicies? This didn't seem to be a problem with my older versions of Logstash and Elasticsearch.

Thanks!

Tony

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/285fb32b-abf5-4d42-94f2-7ca58c75dcf7%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/285fb32b-abf5-4d42-94f2-7ca58c75dcf7%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/C74AC175-77FA-4433-B698-7E6CB264AC4C%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.