# OOM ElasticSearch OUTPUT

**URL:** <https://discuss.elastic.co/t/oom-elasticsearch-output/132158>\
**Category:** Logstash\
**Created:** [May 16, 2018, 2:50pm UTC](https://discuss.elastic.co/t/oom-elasticsearch-output/132158 "2018-05-16T14:50:56Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![aclowkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aclowkey/32/51020_2.png) [@aclowkey](https://discuss.elastic.co/u/aclowkey)\
**Post date:** [May 16, 2018, 2:50pm UTC](https://discuss.elastic.co/t/oom-elasticsearch-output/132158/1 "2018-05-16T14:50:57Z")

</div>

Using LogStash version 6.2.4.  
I have a rather complex pipeline that crashes when I use `elasticsearch{}`output, but works fine when I used `stdout { codec => rubydebug}`.

There are around 115k document - also quite simple - being outputted, almost no free text, mostly attribute values.

I watched `/\_node/stats/event and I saw it just have out: 0, all of a sudden crash. Not that it ramped up or something, just immediately crashed.

I tired running with -b 250, -b 125, b -75

**i tried setting -Xms and -Xmx to 3g and it didn't crash. but I'm not sure this should crash becasue there's too much data. Can I throttle this plugin? change batch size perhaps or workers. I'm lost.**

The output configuration:

```
elasticsearch {
            hosts => "localhost:9200"
            user => "<USER>"
            password => "<PASSWORD>"

            action => update
            index => <index>
            document_id => "%{id}"

            scripted_upsert => true
            script => '
           ctx._source["dummy"] = "dummy" // Just so the script will be compiled
            '
        }

```

I don't get much logs, but here's what I have

```
java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid30485.hprof ...
Heap dump file created [1499118783 bytes in 9.915 secs]
2018-05-16 16:21:34 +0300: Listen loop error: #<IOError: closed stream>
org/jruby/RubyIO.java:3405:in `select'
/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/puma-2.16.0-java/lib/puma/server.rb:322:in `handle_servers'
/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/puma-2.16.0-java/lib/puma/server.rb:296:in `block in run'
2018-05-16 16:21:34 +0300: Listen loop error: #<IOError: closed stream>
org/jruby/RubyIO.java:3405:in `select'
/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/puma-2.16.0-java/lib/puma/server.rb:322:in `handle_servers'
/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/puma-2.16.0-java/lib/puma/server.rb:296:in `block in run'
2018-05-16 16:21:34 +0300: Listen loop error: #<IOError: closed stream>
org/jruby/RubyIO.java:3405:in `select'

```

I tried profiling with VisualVM:  
[![Heap space](https://i.stack.imgur.com/sLYzl.png)](https://i.stack.imgur.com/sLYzl.png)  
[![CPU](https://i.stack.imgur.com/EXIL0.png)](https://i.stack.imgur.com/EXIL0.png)  
[![Memory sampling](https://i.stack.imgur.com/KSdW5.png)](https://i.stack.imgur.com/KSdW5.png)

---

<div class="post-metadata">

**Author:** ![aclowkey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aclowkey/32/51020_2.png) [@aclowkey](https://discuss.elastic.co/u/aclowkey)\
**Post date:** [May 22, 2018, 6:19am UTC](https://discuss.elastic.co/t/oom-elasticsearch-output/132158/2 "2018-05-22T06:19:55Z")

</div>

up up up up

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 19, 2018, 6:19am UTC](https://discuss.elastic.co/t/oom-elasticsearch-output/132158/3 "2018-06-19T06:19:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.