# Index key XXX did not match any of the cached resources (intermittent issue)

**URL:** <https://discuss.elastic.co/t/index-key-xxx-did-not-match-any-of-the-cached-resources-intermittent-issue/318219>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [November 4, 2022, 9:07pm UTC](https://discuss.elastic.co/t/index-key-xxx-did-not-match-any-of-the-cached-resources-intermittent-issue/318219 "2022-11-04T21:07:21Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![vddmpo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vddmpo/32/112968_2.png) [@vddmpo](https://discuss.elastic.co/u/vddmpo)\
**Post date:** [November 4, 2022, 9:07pm UTC](https://discuss.elastic.co/t/index-key-xxx-did-not-match-any-of-the-cached-resources-intermittent-issue/318219/1 "2022-11-04T21:07:21Z")

</div>

Hi,

There is intermittent problem with "add\_kubernetes\_metadata" (resource type: pod) when the log is not enriched. The debug logs show the following:

```auto
{"log.level":"debug","@timestamp":"2022-11-04T18:57:06.524Z","log.logger":"kubernetes","log.origin":{"file.name":"add_kubernetes_metadata/matchers.go","file.line":92},"message":"Incoming log.file.path value: /var/log/pods/xxx-yyy-zzz_management-c98dbbf5d-ktg6c_2198106a-d4a9-46f9-91f3-1fd068be5688/xxx-yyy-management/2.log","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-04T18:57:06.524Z","log.logger":"kubernetes","log.origin":{"file.name":"add_kubernetes_metadata/matchers.go","file.line":125},"message":"Using pod uid: 2198106a-d4a9-46f9-91f3-1fd068be5688","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-04T18:57:06.524Z","log.logger":"kubernetes","log.origin":{"file.name":"add_kubernetes_metadata/kubernetes.go","file.line":275},"message":"Using the following index key 2198106a-d4a9-46f9-91f3-1fd068be5688","service.name":"filebeat","libbeat.processor":"add_kubernetes_metadata","ecs.version":"1.6.0"}
{"log.level":"debug","@timestamp":"2022-11-04T18:57:06.524Z","log.logger":"kubernetes","log.origin":{"file.name":"add_kubernetes_metadata/kubernetes.go","file.line":278},"message":"Index key 2198106a-d4a9-46f9-91f3-1fd068be5688 did not match any of the cached resources","service.name":"filebeat","libbeat.processor":"add_kubernetes_metadata","ecs.version":"1.6.0"}

```

Using `kubectl get pods -n $(echo $i | cut -d":" -f1) $(echo $i | cut -d":" -f2) -o jsonpath='{.metadata.uid}'` I'm able to find the pod with the UID above. In Kibana I see 90% of events with kubernetes metadata, but the rest events are not enriched.

How can I investigate deeper what is happening with `kubernetes.go`?

Btw, the POD mentioned above is not a new one and it is up and running some weeks already (and healthy).

Filebeat versions I tried: v8.4.1 and v8.5.0.  
Kubernetes v1.24.2.  
Container Runtime: containerd (1.6.8-1).

Filebeat configuration:

```auto
        filebeat:
          registry:
            flush: 10s
          inputs:
            - type: container
              scan_frequency: 1s
              close_inactive: 2h
              ingore_older: 72h
              harvester_buffer_size: 65536
              exclude_files: ['.gz$']
              exclude_lines: ["^\\s+[\\-`('.|_]"]
              paths:
                - "/var/log/pods/*/*/*"
              processors:
              - add_kubernetes_metadata:
                  host: ${NODE_NAME}
                  default_indexer.enabled: false
                  default_matchers.enabled: false
                  indexers:
                    - pod_uid:
                  matchers:
                    - logs_path:
                        logs_path: '/var/log/pods/'
                        resource_type: 'pod'

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 2, 2022, 11:07pm UTC](https://discuss.elastic.co/t/index-key-xxx-did-not-match-any-of-the-cached-resources-intermittent-issue/318219/2 "2022-12-02T23:07:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.