# Filebeat OOM

**URL:** <https://discuss.elastic.co/t/filebeat-oom/146736>\
**Category:** Beats\
**Created:** [August 30, 2018, 3:39pm UTC](https://discuss.elastic.co/t/filebeat-oom/146736 "2018-08-30T15:39:08Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![javadevmtl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javadevmtl/32/45613_2.png) [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\
**Post date:** [August 30, 2018, 3:39pm UTC](https://discuss.elastic.co/t/filebeat-oom/146736/1 "2018-08-30T15:39:08Z")

</div>

Hi, running filebeat 6.2.2

Today I noticed filebeat was going bonkers... In /var/log/messages I see allot of these...

> Aug 30 13:30:43 master-0002 kernel: Out of memory: Kill process 19154 (filebeat) score 818 or sacrifice child  
> Aug 30 13:31:40 master-0002 filebeat: fatal error: runtime: out of memory

The metrics output (as you can see it's not monitoring many files:

`2018-08-30T15:37:15.442Z	INFO	[monitoring]	log/log.go:124	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":31800,"time":31803},"total":{"ticks":58140,"time":58143,"value":58140},"user":{"ticks":26340,"time":26340}},"info":{"ephemeral_id":"2e507519-2788-4e40-b4e4-3964f97af9c5","uptime":{"ms":38552}},"memstats":{"gc_next":13559486432,"memory_alloc":9792227856,"memory_total":21843088664,"rss":15713218560}},"filebeat":{"events":{"added":2,"done":2},"harvester":{"open_files":1,"running":1,"started":1}},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"kafka"},"pipeline":{"clients":2,"events":{"active":0,"filtered":2,"total":2}}},"registrar":{"states":{"current":1,"update":2},"writes":2},"system":{"cpu":{"cores":4},"load":{"1":0.66,"15":0.27,"5":0.36,"norm":{"1":0.165,"15":0.0675,"5":0.09}}}}}}`

The config:

```
filebeat.prospectors:
- type: log
  paths:
    - /var/lib/mesos/slave/slaves/*/frameworks/*/executors/*/runs/latest/stdout*
    - /var/lib/mesos/slave/slaves/*/frameworks/*/executors/*/runs/latest/stderr*
  exclude_files: ["stdout.logrotate.state", "stdout.logrotate.conf", "stderr.logrotate.state", "stderr.logrotate.conf"]
  fields:
    source_type: "framework"
  fields_under_root: true
  tail_files: false

- type: log
  paths:
    - /var/log/mesos/*.log
    - /var/log/dcos/dcos.log
  fields:
    source_type: "dcos"
  fields_under_root: true
  tail_files: false

output.kafka:
  hosts: ["xxxxxx:9092"]

  version: 0.11.0.0

  topic: "app-logs"
  partition.round_robin:
    reachable_only: true

  required_acks: -1
  max_message_bytes: 2000000

```

---

<div class="post-metadata">

**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\
**Post date:** [August 31, 2018, 5:13pm UTC](https://discuss.elastic.co/t/filebeat-oom/146736/2 "2018-08-31T17:13:10Z")

</div>

@javadevmtl

You are right, by looking at the metric output you only have 1 file open, if you look at your filebeat log is there any stacktrace present? Is Filebeat generating any errors in the log?

I see that you are running 6.2.2, I know that we have fixed a few leaks in 6.3 and also in 6.4 did you try with the latest release of filebeat if you are running into the same issue?

---

<div class="post-metadata">

**Author:** ![javadevmtl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javadevmtl/32/45613_2.png) [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\
**Post date:** [August 31, 2018, 7:33pm UTC](https://discuss.elastic.co/t/filebeat-oom/146736/3 "2018-08-31T19:33:01Z")

</div>

Unfortunately I didn't catch the log while it happened and I restarted since then...

---

<div class="post-metadata">

**Author:** ![javadevmtl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javadevmtl/32/45613_2.png) [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\
**Post date:** [September 4, 2018, 2:01pm UTC](https://discuss.elastic.co/t/filebeat-oom/146736/4 "2018-09-04T14:01:24Z")

</div>

Well I'm running Elastic 6.2.2. Can I run a newer filebeat? We are using kafka as the queue between the 2.

---

<div class="post-metadata">

**Author:** ![javadevmtl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javadevmtl/32/45613_2.png) [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\
**Post date:** [September 11, 2018, 6:47pm UTC](https://discuss.elastic.co/t/filebeat-oom/146736/5 "2018-09-11T18:47:15Z")

</div>

@pierhugues I'm assuming I can use 6.4.0 Filebeat with Elastic 6.2.4 since I have Kafka in the middle?

---

<div class="post-metadata">

**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\
**Post date:** [September 11, 2018, 7:45pm UTC](https://discuss.elastic.co/t/filebeat-oom/146736/6 "2018-09-11T19:45:30Z")

</div>

Yes you should be able to use it in that scenario.

---

<div class="post-metadata">

**Author:** ![javadevmtl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javadevmtl/32/45613_2.png) [@javadevmtl](https://discuss.elastic.co/u/javadevmtl)\
**Post date:** [September 11, 2018, 7:47pm UTC](https://discuss.elastic.co/t/filebeat-oom/146736/7 "2018-09-11T19:47:13Z")

</div>

Ok I will have to test and eventually deploy it... Thanks.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 9, 2018, 9:47pm UTC](https://discuss.elastic.co/t/filebeat-oom/146736/8 "2018-10-09T21:47:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
