Hints based autodiscovery not working as described in documentation

Elastic Stack Beats
1

I'm following documentation on hints based autodiscovery. I want to use annotations to control which pods logs are being discovered. When I use example config like this, then all containers in cluster gets discovered and all logs are sent to Elasticsearch:

filebeat.autodiscover:
  providers:
    - type: kubernetes
      hints.enabled: true
      hints.default_config:
        type: container
        paths:
          - /var/log/containers/*-${data.container.id}.log

So following documentation I disable default config:

filebeat.autodiscover:
  providers:
    - type: kubernetes
      hints.enabled: true
      hints.default_config.enabled: false

And annotate one pod:

co.elastic.logs/enabled: true

I can see in filebeat logs that the pod gets discovered, but no logs are being sent to Elasticsearch.
What am I missing? Do I need to add additional annotations?

2

Your example does not show a full configuration so it is impossible to know if you doing all that is needed.

In this context filebeat does the autodiscovery and collects the input, then it needs to send it to Logstash and Logstash to ES.

Take a look at this project and you will be fine to connect it all together

Test on local instance like on minikube or similar. I use docker-compose so i cannot confirm about kubernetes but it should work if all of the above is connected well

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.