Hi,
Since yesterday a problem has shown itself and I'm looking to understand the implications and the best approach to a fix.
See this exception header below:
[2015-12-03 08:26:32,460][DEBUG][action.admin.indices.mapping.put] [master_001] failed to put mappings on indices [[test-2015.12.03]], type [unknown] java.lang.IllegalArgumentException: Mapper for [_uid] conflicts with existing mapping in other types: [mapper [_uid] cannot be changed from type [_uid] to [string]] at org.elasticsearch.index.mapper.FieldTypeLookup.checkCompatibility(FieldTypeLookup.java:117)
This seems to be down to a log source having a bunch of field names prepended with underscores. The route the message takes to elasticsearch is logstash shipper => logstash agent => elasticsearch.
It would seem that one of the fields that team are using in their log format clashes with elasticsearch's internal field definitions and that the dynamic mapping generated by logstash is invalid.
Some digging through the elasticsearch docs lead me here:
https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-fields.html
Firstly, this raises a question in my mind regarding logstash. Shouldn't logstash's dynamic mapping be aware of elasticsearch internal field and type defintions and prevent mapping errors of this type?
Secondly, must I strictly avoid sending data into logstash with field names with leading underscores in case they clash with meta-fields etc? If so, what is the best way to handle that? By filtering and stripping leading underscores from field names or by mandating that developers change their field naming?
Regards,
David