# Elasticsearch rejecting records because of parsing error **URL:** \ **Category:** Elasticsearch\ **Created:** [May 7, 2019, 6:10pm UTC](https://discuss.elastic.co/t/elasticsearch-rejecting-records-because-of-parsing-error/180040 "2019-05-07T18:10:46Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![vijayakrishna.rg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vijayakrishna.rg/32/36798_2.png) [@vijayakrishna.rg](https://discuss.elastic.co/u/vijayakrishna.rg)\ **Post date:** [May 7, 2019, 6:10pm UTC](https://discuss.elastic.co/t/elasticsearch-rejecting-records-because-of-parsing-error/180040/1 "2019-05-07T18:10:47Z")
Elastic version: 6.7.0 Kibana version: 6.7.0 Source to send data to elasticsearch: kubernetes, fluentd I am having an issue while parsing message and forwarding from kubernetes cluster to elasticsearch it's rejecting all the records because of parsing issue, almost 1 million records got rejected by elasticsearch. added below parameter into template for index: **"index.mapping.ignore\_malformed": true** ISSUE: [2019-05-07T17:52:22,596][DEBUG][o.e.a.b.TransportShardBulkAction] [elastic1] [k8s-pi-2019.05.07][2] failed to execute bulk item (index) index {[k8s-pi-2019.05.07][fluentd][rnZsk2oBoyEjHE85xxtC], source[{"@severity":"DEBUG","@timestamp":"2019-05-07T17:52:17.537Z","@traceId":["xxx-4168-9b4c-7f17d8f6d025","PROCESSOR-CONSUME-0"],"class":"com.commons.stream.eventhubs.ProcessorComponent","file":"ProcessorComponent","line":85,"msg":{"@msg":"Consumed a message from an EventHubs queue","EventAction":{"BAD1":"EventAction(addressCreated)"},"app":"processor-update","nodeIp":"10.10.10.10","nodeName":"","podName":"processor-update-xxx-2s2qd","region":"japan"},"log":"{"@severity":"DEBUG","@timestamp":"2019-05-07T17:52:17.537Z","@traceId":["4147717c-bf50-4168-asrd-7f17d8f6d025","PROCESSOR-CONSUME-0"],"class":"com.commons.stream.eventhubs.ProcessorComponent","file":"ProcessorComponent","line":85,"msg":{"@msg":"Consumed a message from an EventHubs queue","**EventAction":{"BAD1":"EventAction(addressCreated)"}**,"app":"processor-update","nodeIp":"10.10.10.10","nodeName":"","podName":"processor-update-86cc9687bf-xxxx","region":"japan"}}\n","stream":"stdout","docker":{"container\_id":"2d9b63f5234234sdfsd35sdfsdfdf0da010da173806cb4ff2222b8754d2644f073b"},"kubernetes":{"container\_name":"processor-update","namespace\_name":"xx","pod\_name":"processor-update-86cc9687bf-ggtgg","pod\_id":"ae262158-xxx-11e9-xxx-000d3a34cdc9","labels":{"app":"processor-update","checksum/config":"6723432404290xxxxxcdd0440f5831ce07cbaf0faa42bc7c","pod-template-hash":"4277524369"},"host":"test-worker-1","master\_url":"[https://10.10.10.10:443/api](https://10.10.10.10:443/api)"}}]} **org.elasticsearch.index.mapper.MapperParsingException: failed to parse field [msg.EventAction] of type [text] in document with id 'rnZsk2oBoyEjHE85zrtC'** i have included malformed\_ignore parameter while creating index, still getting an above issue. is there a way we can include any kind of mapping in index template to ignore above issue at least [msg.EventAction] field., Please suggest me, Thanks. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 8, 2019, 6:07am UTC](https://discuss.elastic.co/t/elasticsearch-rejecting-records-because-of-parsing-error/180040/2 "2019-05-08T06:07:41Z")
If you look in the documentation [there are limits to how this can be used when it comes to JSON objects](https://www.elastic.co/guide/en/elasticsearch/reference/7.0/ignore-malformed.html#json-object-limits), which is the case you seem to have here as the field is mapped as string and you send in an object. ---
**Author:** ![vijayakrishna.rg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vijayakrishna.rg/32/36798_2.png) [@vijayakrishna.rg](https://discuss.elastic.co/u/vijayakrishna.rg)\ **Post date:** [May 21, 2019, 10:59pm UTC](https://discuss.elastic.co/t/elasticsearch-rejecting-records-because-of-parsing-error/180040/3 "2019-05-21T22:59:41Z")
Hi @Christian_Dahlqvist, i have update index settings with "ignore\_malformed" : "true", i think it supposed to ignore these mapping errors if we set it up, but still am getting above issue, any suggestion? GET k8s-pi-2019.05.21/\_settings { "k8s-pi-2019.05.21" : { "settings" : { "index" : { "lifecycle" : { "name" : "k8s-logs", "rollover\_alias" : "k8s" }, "mapping" : { "total\_fields" : { "limit" : "10000" }, "ignore\_malformed" : "true" }, "number\_of\_shards" : "5", "provided\_name" : "k8s-pi-2019.05.21", "creation\_date" : "1558396801813", "number\_of\_replicas" : "1", "uuid" : "7Igc6Z-cRmGlxHKzOf10Aw", "version" : { "created" : "6070099" } } } } } ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 22, 2019, 1:11am UTC](https://discuss.elastic.co/t/elasticsearch-rejecting-records-because-of-parsing-error/180040/4 "2019-05-22T01:11:18Z")
The error you showed earlier is not covered by the ignore malformed flag. This is described in the docs I linked to. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 19, 2019, 1:11am UTC](https://discuss.elastic.co/t/elasticsearch-rejecting-records-because-of-parsing-error/180040/5 "2019-06-19T01:11:29Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.