# Filebeat \[7.15.0\] hints based discovery does nothing

**URL:** <https://discuss.elastic.co/t/filebeat-7-15-0-hints-based-discovery-does-nothing/287045>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [October 19, 2021, 3:13am UTC](https://discuss.elastic.co/t/filebeat-7-15-0-hints-based-discovery-does-nothing/287045 "2021-10-19T03:13:27Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![gwvandesteeg](https://avatars.discourse-cdn.com/v4/letter/g/839c29/32.png) [@gwvandesteeg](https://discuss.elastic.co/u/gwvandesteeg)\
**Post date:** [October 19, 2021, 3:13am UTC](https://discuss.elastic.co/t/filebeat-7-15-0-hints-based-discovery-does-nothing/287045/1 "2021-10-19T03:13:27Z")

</div>

Continuing the discussion from [Filebeat doesn't send logs to Elasticsearch](https://discuss.elastic.co/t/filebeat-doesnt-send-logs-to-elasticsearch/259228/8):

We're having the exact same problem as the above which doesn't appear to have ever been resolved.  
Using filebeat 7.15.0, exact same config as above, running on `kind`, we have a very simple test setup.

- filebeat configured as above to write to logstash
- logstash configured to write to stdout
- podinfo installed

To reproduce:

1. Start kind (or whatever k8s cluster you want to use)
2. Install podinfo using helm3 `helm upgrade --install --wait backend -f podinfo-values.yaml --set redis.enabled=true podinfo/podinfo` where the `podinfo-values.yaml` contains:

```auto
---
podAnnotations:
  co.elastic.logs/enabled: "true"
  co.elastic.logs/json.keys_under_root: "false"
  co.elastic.logs/json.message_key: message
  co.elastic.logs/json.overwrite_keys: "true"

```

1. Install logstash via helm with the below config `helm upgrade --install logstash elastic/logstash -f logstash-values.yaml`, the content of `logstash-values.yaml` below.

```auto
---
fullnameOverride: logstash
logstashConfig:
  logstash.yml: |
    http.host: 0.0.0.0
    pipeline.ecs_compatibility: "v1"
    config.reload.automatic: true
    config.reload.interval: 900s

logstashPipeline:
  logstash.conf: |
    input {
      beats {
        port => 5044
      }
    }
    filter {
      # don't need these
      if [kubernetes][namespace] == "kube-system" {
        drop { }
      }
      # only care about podinfo
      if [kubernetes][container][name] in ["filebeat", "logstash"] {
        drop { }
      }
    }
    output {
      stdout { }
    }

```

1. Install filebeat via helm using `helm upgrade --install filebeat ./helm-charts/filebeat -f filebeat-values.yaml --set imageTag=7.15.0` where `filebeat-values.yaml` contains:

```auto
---
fullnameOverride: filebeat
daemonset:
  annotations:
    co.elastic.logs/enabled: "false"
  enabled: true
  filebeatConfig:
    filebeat.yml: |
      # hints based autodiscover from the kubernetes API
      filebeat.autodiscover:
        providers:
          - type: kubernetes
            node: ${NODE_NAME}
            hints.enabled: true
            hints.default_config:
              enabled: true
              type: container
              paths:
                - /var/log/containers/*${data.kubernetes.container.id}.log

      # output to logstash
      output.logstash:
        enabled: true
        hosts: ["logstash:5044"]
        loadbalance: true

deployment:
  annotations:
    co.elastic.logs/enabled: "false"
  enabled: false

```

Logs for podinfo don't come through to logstash, just sits there idle. Filebeat generates a metric "bucketload" of logs since it is parsing its own multiline logs and feeding back on itself (meaning the hint to disable it doesn't seem to work).

Anyone got any suggestions? We've gone through many different permutations of configurations and sofar haven't found a single one that works.

BTW: `podinfo` generated logs look like the below as per: `kubectl logs`

```auto
{"level":"info","ts":"2021-10-19T02:02:02.563Z","caller":"podinfo/main.go:151","msg":"Starting podinfo","version":"6.0.0","revision":"","port":"9898"}
{"level":"info","ts":"2021-10-19T02:02:02.566Z","caller":"api/server.go:252","msg":"Starting HTTP Server.","addr":":9898"}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 16, 2021, 5:13am UTC](https://discuss.elastic.co/t/filebeat-7-15-0-hints-based-discovery-does-nothing/287045/2 "2021-11-16T05:13:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.