Where is my infrastructure data?

[ethrbunny]

Hmm.. another puzzle..

FATAL  ValidationError: child "xpack" fails because [child "infra" fails because [child "sources" fails because [child "default" fails because [child "fields" fails because ["hostname" is not allowed]]]]]

If my index follow 'metricbeat-' what would I set your xpack value(s) to?

[weltenwort (Felix Stürmer)]

oh, looks like I made a mistake in my config snippet - it should be host instead of hostname:

xpack.infra:
  sources:
    default:
      fields:
        host: 'beat.hostname.keyword'

sorry about that

[ethrbunny]

That's seems to be working. Thanks for your help!

[ethrbunny]

Sorry - one more thing: what's the config to get the k8 monitoring working?

[weltenwort (Felix Stürmer)]

The k8s metrics rely on the kubernetes metricbeat module and the add_kubernetes_metadata processor being enabled: https://www.elastic.co/guide/en/infrastructure/guide/current/install-infrastructure-monitoring.html#_which_modules_and_configuration_options_do_i_enable

[ethrbunny]

I updated my daemonset definition to include the "processors" entries from the link you provided. When I redeployed it these log entries started showing up:

"metricbeat":{"kubernetes":{"container":{"events":3,"failures":3},"node":{"events":3,"failures":3},"pod":{"events":3,"failures":3},"system":{"events":3,"failures":3},"volume":{"events":3,"failures":3}}

BTW: possible doc bug: in the "kubernetes" section for the same link - you still have the link for "filebeat docker input". Not sure if this is intentional or not.

[weltenwort (Felix Stürmer)]

@exekias any idea about these messages?

possible doc bug: in the "kubernetes" section for the same link - you still have the link for "filebeat docker input"

This is intentional since the even the logs for k8s containers are read using the docker input and correlated using the k8s metadata processor.