# Filebeat not collecting logs for hints based autodiscover in kubernetes

**URL:** <https://discuss.elastic.co/t/filebeat-not-collecting-logs-for-hints-based-autodiscover-in-kubernetes/342134>\
**Category:** Beats\
**Tags:** docker, filebeat\
**Created:** [September 1, 2023, 3:50pm UTC](https://discuss.elastic.co/t/filebeat-not-collecting-logs-for-hints-based-autodiscover-in-kubernetes/342134 "2023-09-01T15:50:28Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![sayantanvlabs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sayantanvlabs/32/125200_2.png) [@sayantanvlabs](https://discuss.elastic.co/u/sayantanvlabs)\
**Post date:** [September 1, 2023, 3:50pm UTC](https://discuss.elastic.co/t/filebeat-not-collecting-logs-for-hints-based-autodiscover-in-kubernetes/342134/1 "2023-09-01T15:50:28Z")

</div>

I am trying to get logs from pods annotated with `co.elastic.logs/enabled: "true"` into elasticsearch using filebeat. But it is not working. I am attaching the configurations I am using, please let me know if I am missing something.

```auto

# Setup Logstash.
cat <<EOF | kubectl apply -f -
apiVersion: logstash.k8s.elastic.co/v1alpha1
kind: Logstash
metadata:
  name: jaegerpoc-elastic
  namespace: elastic-system
spec:
  count: 1
  elasticsearchRefs:
    - name: jaegerpoc-elastic
      clusterName: jes
  version: 8.9.1
  pipelines:
    - pipeline.id: main
      config.string: |
        input {
          beats {
            port => 5044
          }
        }
        output {
          elasticsearch {
            hosts => "http://jaegerpoc-elastic-es-http.elastic-system.svc:9200"
            user => "${JES_ES_USER}"
            password => "${JES_ES_PASSWORD}"
            index => "logs"
          }
        }
  services:
    - name: beats
      service:
        spec:
          type: ClusterIP
          ports:
            - port: 5044
              name: "filebeat"
              protocol: TCP
              targetPort: 5044
EOF

```

```auto
# Setup FileBeat
cat <<EOF | kubectl apply -f -
apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: jaegerpoc-elastic
  namespace: elastic-system
spec:
  type: filebeat
  version: 8.9.1
  elasticsearchRef:
    name: jaegerpoc-elastic
  config:
    filebeat.autodiscover:
      providers:
        - type: kubernetes
          node: "minikube"
          hints:
            enabled: true
            default_config:
              enabled: false
              type: container
              paths:
                - "/var/log/containers/*\${data.kubernetes.container.id}.log"
    output.elasticsearch:
      enabled: false
    output.logstash:
      hosts: ["jaegerpoc-elastic-ls-beats.elastic-system.svc.cluster.local:5044"]
  daemonSet:
    podTemplate:
      spec:
        serviceAccountName: jaegerpoc-elastic-beat-sa
        automountServiceAccountToken : true
        dnsPolicy: ClusterFirstWithHostNet
        hostNetwork: true
        securityContext:
          runAsUser: 0
        containers:
        - name: filebeat
          volumeMounts:
          - name: varlogcontainers
            mountPath: /var/log/containers
          - name: varlogpods
            mountPath: /var/log/pods
          - name: varlibdockercontainers
            mountPath: /var/lib/docker/containers
        volumes:
        - name: varlogcontainers
          hostPath:
            path: /var/log/containers
        - name: varlogpods
          hostPath:
            path: /var/log/pods
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
EOF

```

```auto
cat <<EOF | kubectl apply -f -
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jaegerpoc-elastic-beat-sa
  namespace: elastic-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: elastic-beat-autodiscover
rules:
- apiGroups:
  - ""
  resources:
  - nodes
  - namespaces
  - events
  - pods
  - replicasets
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - extensions
  - apps
  resources:
  - deployments
  - replicasets
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: elastic-beat-autodiscover-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: elastic-beat-autodiscover
subjects:
- kind: ServiceAccount
  name: jaegerpoc-elastic-beat-sa
  namespace: elastic-system
EOF

```

Now my pods are being deployed with the logs enabled annotation. And I can confirm it is producing the logs.But it's not coming to elasticsearch.  
Although if I use `filebeat.inputs` instead of autodiscover, I am getting logs for all the pods

```auto
filebeat.inputs:
    - type: container
      paths:
      - /var/log/containers/*.log

```

 ![Screenshot from 2023-09-01 21-17-53](https://us1.discourse-cdn.com/elastic/original/3X/2/6/261262737af0eac745a7265ed59821cbd39290d1.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 29, 2023, 5:50pm UTC](https://discuss.elastic.co/t/filebeat-not-collecting-logs-for-hints-based-autodiscover-in-kubernetes/342134/2 "2023-09-29T17:50:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.