# Cannot collect logs from kubernetes with containerd runtimes

**URL:** <https://discuss.elastic.co/t/cannot-collect-logs-from-kubernetes-with-containerd-runtimes/326678>\
**Category:** Elastic Agent\
**Tags:** filebeat\
**Created:** [February 28, 2023, 10:19am UTC](https://discuss.elastic.co/t/cannot-collect-logs-from-kubernetes-with-containerd-runtimes/326678 "2023-02-28T10:19:57Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![venturieffect](https://avatars.discourse-cdn.com/v4/letter/v/8797f3/32.png) [@venturieffect](https://discuss.elastic.co/u/venturieffect)\
**Post date:** [February 28, 2023, 10:19am UTC](https://discuss.elastic.co/t/cannot-collect-logs-from-kubernetes-with-containerd-runtimes/326678/1 "2023-02-28T10:19:58Z")

</div>

Hi everyone  
We deployed elastic agents with kubernetes integrations on newly installed kubernetes clusters 1.24. We noticed some missing logs and investigated the agents logs and status: we have this kind of error for every log file

```auto
11:10:17.345
elastic_agent.filebeat
[elastic_agent.filebeat][debug] Incoming log.file.path value: /var/log/containers/elastic-agent-f5tqx_monitoring_elastic-agent-c38bb0fa3843aed8800b5bbe230984693f6b1860f4e46c6910c46a4fcbd6dbb2.log
11:10:17.345
elastic_agent.filebeat
[elastic_agent.filebeat][debug] log.file.path value does not contain matcher's logs_path '/var/lib/docker/containers/', skipping...
11:10:17.345
elastic_agent.filebeat
[elastic_agent.filebeat][debug] No container match string, not adding kubernetes data

```

and the detailed `elastic-agent status --output yaml` command returns a lot of this. one for each log file

```yaml
  - unit_id: filestream-default-filestream-container-logs-00222d86-dbb3-401d-8d4b-c00aeddc4fc1-kubernetes-40aada03-ef38-4dde-a6e9-68e1d06dd132.api-server
    unit_type: 0
    state: 4
    message: 'Failed: pid ''11842'' exited with code ''-1'''

```

Since kubernetes 1.24 uses containerd and not docker anymore, how should we configure the kubernetes integration? are those filebeat crashes related to the missing /var/lib/docker folder?

I tried every combination of processor configuration but seems it's been overridden by the main filebeat configuration which I have not accesso to, being within elastic agent.  
can you help me please?

---

<div class="post-metadata">

**Author:** ![MichelLaterman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michellaterman/32/110221_2.png) [@MichelLaterman](https://discuss.elastic.co/u/MichelLaterman)\
**Post date:** [March 1, 2023, 9:17pm UTC](https://discuss.elastic.co/t/cannot-collect-logs-from-kubernetes-with-containerd-runtimes/326678/2 "2023-03-01T21:17:40Z")

</div>

Hi @venturieffect,  
Is this the same issue as [Elastic Agent 8.6.x standalone deployment in Kubernetes doesn't start monitoring new pods until agent restart · Issue #2269 · elastic/elastic-agent · GitHub](https://github.com/elastic/elastic-agent/issues/2269)?

---

<div class="post-metadata">

**Author:** ![venturieffect](https://avatars.discourse-cdn.com/v4/letter/v/8797f3/32.png) [@venturieffect](https://discuss.elastic.co/u/venturieffect)\
**Post date:** [March 2, 2023, 8:46am UTC](https://discuss.elastic.co/t/cannot-collect-logs-from-kubernetes-with-containerd-runtimes/326678/3 "2023-03-02T08:46:17Z")

</div>

Hi Michel  
It may be the case, yes.  
We noticed two separate issues:

- On one kubernetes cluster the filestream units are constantly crashing as showed in my opening post
- on basically all other kubernetes clusters the filestream units are healty but the count is way less than the actual number of pods in the node. I also experienced what your github issue says: if the pods are restarted, they now appears among the filestream units

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 30, 2023, 8:47am UTC](https://discuss.elastic.co/t/cannot-collect-logs-from-kubernetes-with-containerd-runtimes/326678/4 "2023-03-30T08:47:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.