I was looking at some of my logs this morning, and across all of my nodes, I am seeing an occasional log line that is consitent:
[2017-07-05T11:00:04,760][DEBUG][o.e.a.b.TransportShardBulkAction] [ES102] [log-source-2017.07.05.11][3] failed to execute bulk item (index) BulkShardRequest [[log-source-2017.07.05.11][3]] containing [91] requests
java.lang.IllegalArgumentException: mapper [source_geoip.dma_code] of different type, current_type [text], merged_type [long]
This is happening across all of the index types, but it is a rare occasion. So maybe 5-6 times in the last few days, per node. Considering we are running the geoip call 30k times a second, it is rare. But it does exist.
It looks like the call to geoip sometimes produces a value that is not a 'long' value for the dma_code value inside of the geoip value, sometimes producing a 'text'. This, in return, is causing a bulk submit to fail.
Has anyone else seen this, and how do we go about confirming this as a bug?