I am currently evaluating different approaches to filter filter for deployments / statefulsets and show the child pods and containers including their metrics.
I noticed that in all metricsets having the prefix state_ the kubernetes labels are shown like this:
For pods they are stored below kubernetes.pod._module.labels.*.
Could you please tell me:
why did you choose different fields for storing the same logical data?
Is there any method available in kibana to filter for joining / combining both fields to a container? Creating a manual filter with an ES-Query where I query both fields with or condition is not an option. That is unable to be used by normal users with dynamically changing filter queries.
Is there any option in metricbeat NOT to add ._module to the labels? I would like to see all kubernetes labels in kubernetes.labels - regardless of the module which captured it / which kubernetes resource is affected.
If currently no solution is available my only idea is writing a logstash ruby filter which will dynamically clone the the field kubernetes.<module>._module.labels.* to kubernetes.labels.*
I am open for any solutions. So hopefully I have just missed some easy option, because I don't think that my usecase is so special.
unfortunately I'm unable to tell you why this is designed as is, I'm leaving that question open so some of the beats/metricbeats members can clarify. I also wonder why _module is there and if underscore has any special meaning.
I opened an issue I hope will be working if soon if I get no comments preventing it, that would do a more homogeneous use of labels reporting path and some pre-processing filtering: https://github.com/elastic/beats/issues/12938
I haven't tested it, and the documentation doesn't indicate if the fields need to be exact path, can be partials, or if wildcards are allowed ... hope that helps anyway
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.