Change in document _type?

[Badger]

I am ingesting both G1 and J9 garbage collection logs into elasticsearch. Filebeat sends them to kafka, and logstash pulls them in from there. I installed 6.0 GA this morning, and deleted data/nodes/0 before starting es. I am having an issue.

[2017-11-15T12:38:23,158][INFO ][o.e.c.m.MetaDataCreateIndexService] [hostname][logs.g1gc-2017.11] creating index, cause [auto(bulk api)], templates [g1gc], shards [2]/[1], mappings [_default_]
[2017-11-15T12:38:23,292][INFO ][o.e.c.m.MetaDataMappingService] [hostname] [logs.g1gc-2017.11/Be7hUystQYSw-tmIhWjDKg] create_mapping [doc]
[2017-11-15T12:38:23,315][INFO ][o.e.c.m.MetaDataMappingService] [hostname] [logs.g1gc-2017.11/Be7hUystQYSw-tmIhWjDKg] update_mapping [doc]
[...]
[2017-11-15T12:38:30,281][DEBUG][o.e.a.a.i.m.p.TransportPutMappingAction] [hostname] failed to put mappings on indices [[[logs.g1gc-2017.11/Be7hUystQYSw-tmIhWjDKg]]], type [log]
java.lang.IllegalArgumentException: Rejecting mapping update to [logs.g1gc-2017.11] as the final mapping would have more than 1 type: [log, doc]
        at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:494) ~[elasticsearch-6.0.0.jar:6.0.0]

I think the problem is that events from my server (just upgraded to 6.0 GA) are of type doc, and events coming from a colleague's 5.x box are of type log. Did that change in filebeat? I did not find it in a quick review of the release notes.

[ruflin]

Yes, this did change. Elasticsearch 6.0 removed support for types: https://www.elastic.co/guide/en/elasticsearch/reference/master/removal-of-types.html That mean in beats now the type that is used is the default type doc. I assume the file instances you use on the other boxes are before 5.6?

[Badger]

Yes, he runs 5.4 if I recall correctly. He doesn't upgrade every two weeks like I do

I can set document_type on the es output in logstash to fix this.

[ruflin]

Great you have a workaround.