# Filebeat pod continuously restarting : Liveness probe failed: rss\_mem 341245952

**URL:** <https://discuss.elastic.co/t/filebeat-pod-continuously-restarting-liveness-probe-failed-rss-mem-341245952/341980>\
**Category:** Beats\
**Tags:** docker, filebeat\
**Created:** [August 30, 2023, 11:37am UTC](https://discuss.elastic.co/t/filebeat-pod-continuously-restarting-liveness-probe-failed-rss-mem-341245952/341980 "2023-08-30T11:37:35Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sam\_John](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sam_john/32/125104_2.png) [@Sam\_John](https://discuss.elastic.co/u/Sam_John)\
**Post date:** [August 30, 2023, 11:37am UTC](https://discuss.elastic.co/t/filebeat-pod-continuously-restarting-liveness-probe-failed-rss-mem-341245952/341980/1 "2023-08-30T11:37:35Z")

</div>

filebeat pod in a k8s worker node is continuously restarting with following error message when the pods in the worker node increases:

**Error Message from filebeat pod:**  
Warning Unhealthy 44m kubelet, k8s-worker-2 **Liveness probe failed: rss\_mem 341245952**

**1)**  
**I am using following images in filebeat daemon set:**

[docker.elastic.co/beats/filebeat-oss:7.9.0](http://docker.elastic.co/beats/filebeat-oss:7.9.0)  
trustpilot/beat-exporter:0.1.1

**2)**  
**I have tried following workarounds but could not resolve the issue**  
a)

> **[High RSS memory usage due to MADV settings | Filebeat Reference \[8.9\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/madvdontneed-rss.html)**

b)  
Increased memory limit of filebeat container in filebeat daemon set

c)  
upgraded filebeat-oss docker image to 7.10.2

**3)**  
**Filbeat memory and cpu allocation which we are using**

```auto
    resources:
      limits:
        cpu: 100m
        memory: 512Mi
      requests:
        cpu: 100m
        memory: 512Mi

```

**4)**  
**update filebeat livnessprobe**

I can see following livnessprobe in filebeat daemon set, but latest filebeat doesn't use rss memory check in livnessprobe , Can I remove this rss memory check for filebeat oss 7.9.0 ?

```auto
    livenessProbe:
      exec:
        command:
        - /bin/bash
        - -c
        - var=($(curl -s http://localhost:5066/stats?pretty | grep 'rss\|open_files'
          | sed 's/[^0-9]*//g')); echo -e " **rss_mem** ${var[0]} \nopen_files ${var[1]}";
          if ["${var[0]}" -gt " **335544320**" ] || ["${var[1]}" -le "1" ]; then
          exit 1; else exit 0; fi

```

**5)**  
**Update filebeat.yml**

Should I make any configuration change in filebeat.yml file to fix this issue ? We are using default values.  
**for example :**

```auto
bulk_max_size

```

> **[filebeat.reference.yml | Filebeat Reference \[7.9\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/7.9/filebeat-reference-yml.html)**

---

<div class="post-metadata">

**Author:** ![Sam\_John](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sam_john/32/125104_2.png) [@Sam\_John](https://discuss.elastic.co/u/Sam_John)\
**Post date:** [August 30, 2023, 6:55pm UTC](https://discuss.elastic.co/t/filebeat-pod-continuously-restarting-liveness-probe-failed-rss-mem-341245952/341980/2 "2023-08-30T18:55:23Z")

</div>

any clue ?

---

<div class="post-metadata">

**Author:** ![Sam\_John](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sam_john/32/125104_2.png) [@Sam\_John](https://discuss.elastic.co/u/Sam_John)\
**Post date:** [September 1, 2023, 6:28am UTC](https://discuss.elastic.co/t/filebeat-pod-continuously-restarting-liveness-probe-failed-rss-mem-341245952/341980/3 "2023-09-01T06:28:25Z")

</div>

can I remove RSS memory check `-gt "335544320"` from livenessProbe ? Is it safe ?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 29, 2023, 8:28am UTC](https://discuss.elastic.co/t/filebeat-pod-continuously-restarting-liveness-probe-failed-rss-mem-341245952/341980/4 "2023-09-29T08:28:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.