# Logstash OOM kills - k8s - 8.13.0

**URL:** <https://discuss.elastic.co/t/logstash-oom-kills-k8s-8-13-0/358423>\
**Category:** Logstash\
**Created:** [April 29, 2024, 11:39am UTC](https://discuss.elastic.co/t/logstash-oom-kills-k8s-8-13-0/358423 "2024-04-29T11:39:37Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![d4nnyx](https://avatars.discourse-cdn.com/v4/letter/d/e99b99/32.png) [@d4nnyx](https://discuss.elastic.co/u/d4nnyx)\
**Post date:** [April 29, 2024, 11:39am UTC](https://discuss.elastic.co/t/logstash-oom-kills-k8s-8-13-0/358423/1 "2024-04-29T11:39:37Z")

</div>

Hi everyone,

We are using the logging pipeline Filebeat -\> Logstash -\> Dataset/Scalyr.

The issue I'm dealing with is that the Logstash memory keeps increasing until the OOM kills the pod (so we are talking about OS OOM kill, not the heap one).

With default settings, huge heap, low heap, low/high number of workers, and all combinations, the result is always the same.

The interesting thing is, that if I restrict the incoming traffic to a pod, the memory is still kept high. Even when the events are not there anymore and were successfully delivered to the target database.

**Deployment details:**  
Logstash v8.13.0

Additional filters (latest versions):  
logstash-filter-prune  
logstash-filter-json\_encode  
logstash-output-scalyr

Kubernetes version v1.24.12  
JVM min and max set to 50-75% of the physical memory  
Mem: 30Gi  
CPU: 14

Inputs:  
Beats, TCP, Syslog

Filters:  
Mutate, drop, truncate, json, grok

Output:  
Scalyr

Java Native memory tracking output (The "Other" category is quite big)

```auto
jdk/bin/jcmd 1 VM.native_memory summary scale=mb
Defaulted container "logstash" out of: logstash, logstash-exporter
1:

Native Memory Tracking:

(Omitting categories weighting less than 1MB)

Total: reserved=28108MB, committed=26198MB
       malloc: 9838MB #824166
       mmap: reserved=18270MB, committed=16360MB

- Java Heap (reserved=15360MB, committed=15360MB)
                            (mmap: reserved=15360MB, committed=15360MB)

- Class (reserved=1032MB, committed=28MB)
                            (classes #25640)
                            ( instance classes #24548, array classes #1092)
                            (malloc=8MB #125737) (peak=8MB #142766)
                            (mmap: reserved=1024MB, committed=21MB)
                            ( Metadata: )
                            ( reserved=128MB, committed=107MB)
                            ( used=102MB)
                            ( waste=4MB =3.91%)
                            ( Class space:)
                            ( reserved=1024MB, committed=21MB)
                            ( used=17MB)
                            ( waste=4MB =17.81%)

- Thread (reserved=868MB, committed=96MB)
                            (thread #862)
                            (stack: reserved=865MB, committed=93MB)
                            (malloc=1MB #5188) (peak=2MB #6050)
                            (arena=1MB #1722) (peak=63MB #1720)

- Code (reserved=247MB, committed=138MB)
                            (malloc=5MB #13466) (peak=11MB #29022)
                            (mmap: reserved=242MB, committed=133MB)

- GC (reserved=919MB, committed=919MB)
                            (malloc=284MB #84860) (peak=326MB #84115)
                            (mmap: reserved=635MB, committed=635MB)

- Compiler (reserved=11MB, committed=11MB)
                            (malloc=11MB #5957) (peak=11MB #5986)

- Internal (reserved=3MB, committed=3MB)
                            (malloc=3MB #54069) (peak=6MB #82163)

- Other (reserved=9465MB, committed=9465MB)
                            (malloc=9465MB #1358) (peak=9545MB #1361)

- Symbol (reserved=17MB, committed=17MB)
                            (malloc=15MB #505204) (peak=15MB #505205)
                            (arena=2MB #1) (at peak)

- Native Memory Tracking (reserved=13MB, committed=13MB)
                            (tracking overhead=13MB)

- Shared class space (reserved=16MB, committed=12MB)
                            (mmap: reserved=16MB, committed=12MB)

- Arena Chunk (reserved=9MB, committed=9MB)
                            (malloc=9MB #2061) (peak=333MB #9846)

- Tracing (reserved=15MB, committed=15MB)
                            (malloc=15MB #206) (at peak)

- Module (reserved=2MB, committed=2MB)
                            (malloc=2MB #9465) (at peak)

- Synchronization (reserved=1MB, committed=1MB)
                            (malloc=1MB #7670) (peak=1MB #7674)

- Metaspace (reserved=130MB, committed=109MB)
                            (malloc=2MB #3079) (at peak)
                            (mmap: reserved=128MB, committed=107MB)

```

Logstash config

```auto
config.reload.automatic: true
http.host: 0.0.0.0
http.port: 9600
log.format: json
path.config: /usr/share/logstash/pipeline
pipeline.batch.size: 2000
pipeline.ecs_compatibility: disabled
pipeline.workers: 30

```