Filebeat is configured to perform auto-discovery of Docker containers, and ship its logs to logstash. On the logstash side of things however, I see the follow messages for all docker logs:
... "reason"=>"mapper [docker.container.labels.com.docker.swarm.task] of different type, current_type [keyword], merged_type [ObjectMapper]" ...
Thank you for testing Filebeat and providing feedback!
It seems mapping failed due to some of your labels, com.docker.swarm.task is currently holding an object, and a single keyword is being indexed (and failing). From what I see you probably have a set of lables like this:
This is an issue we have recently fixed (yet to be released), and it's related to the dots in label names and how Elasticsearch stores them.
Something you could do as of today is ignoring the com.docker.swarm.task field from Filebeat side, to avoid mapping errors. You can do that by using the drop_field processor:
Correct, on the labels; these are the labels as they are added by Docker Swarm.
Good to hear that this was recently fixed, for the record though - this issue can be reproduced on a single swarm node/single filebeat instance. The value of the label 'com.docker.swarm.task' always triggers this errors.
I have implemented your suggested workaround; which as expected resolves the issue.
Minor note for others who might read this, it should be:
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.