Kubernetes metadata missing from APM documents

#1

Kibana V. 6.5.4:

ES V. 5.6:

APM V 6.6.0:

APM agent nodeJS v 1.12:

**Installed from docker image docker.elastic.co/apm/apm-server:6.6.0 **:

Fresh install

Is there anything special in your setup? Using kubernetes, deploying APM as a Service, default namespace, no customizations except adding kubernetes metadata:

Sample config:

Problem: Kubernetes metadata is not added to the APM traces:

Steps to reproduce:

  1. Launch APM-server using kubectl and this config file:
    https://gist.github.com/digressionary/573939468d9f6d2afe284d2c6014d52b
  2. Configure a node.js app which is running in kubernetes to connect to this APM-server
  3. Hit the node.js app endpoint
  4. the following output is missing kubernetes metadata
    https://gist.github.com/digressionary/3ffbca67eb0940cb2992e5475aace8ab
(Stephen Belanger) #2

v1 of the Node.js agent doesn't report kubernetes data. You'll need to upgrade to v2.

#3

Thank you for your kind response.
I upgraded the APM agent to 2.5.1.

All configuration is the same.

However, I am experiencing the same issue (the metadata is not added).

Could this be a problem specific to running kubernetes as "docker-for-desktop" on a MacBook?

Thankyou,

(Stephen Belanger) #4

Only pod id and name are recorded automatically, and only if they have been found by the container-info module. You can try running that module in your environment to see what it gets, if you like. If that is unable to identify your environment then you will need to supply the data manually via the agent.start(...) options.

#5

Okay I'm a bit confused.

Here, the docs list all the kubernetes fields that are "exported" which I took to mean "added to the document"

https://www.elastic.co/guide/en/apm/server/current/exported-fields-kubernetes-processor.html

However, your answer seems to indicate that only two of these fields would be populated, and only in certain circumstances. Could you clarify?

Thank you.

(Stephen Belanger) #6

That is where data is mapped to in the Elasticsearch documents created by apm-server, if it is given the appropriate input data to include that. It will not record that data itself, it will only map it from the input it gets from agents.