# Issues with Logstash and Elasticsearch Input plugin (Out of Memory)

**URL:** <https://discuss.elastic.co/t/issues-with-logstash-and-elasticsearch-input-plugin-out-of-memory/63717>\
**Category:** Logstash\
**Created:** [October 24, 2016, 8:37am UTC](https://discuss.elastic.co/t/issues-with-logstash-and-elasticsearch-input-plugin-out-of-memory/63717 "2016-10-24T08:37:12Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![fxiger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fxiger/32/12670_2.png) [@fxiger](https://discuss.elastic.co/u/fxiger)\
**Post date:** [October 24, 2016, 8:37am UTC](https://discuss.elastic.co/t/issues-with-logstash-and-elasticsearch-input-plugin-out-of-memory/63717/1 "2016-10-24T08:37:12Z")

</div>

Hi,

ubuntu 16.04 lts  
logstash 2.4.0  
elasticsearch 2.4.1, Build: c67dc32/2016-09-27T18:57:55Z  
JVM: 1.8.0\_91

let's assume, i have 2 systems (A and B). System A is running somewhere and has an ELK stack on top with an index created out of log files. I do not have access to the logs of system A but i'm able to access to ELK stack there and want to "replicate" the index and use it on system B (also ELK stack) for further investigations, enriching of events and stuff.

When i run logstash via shell like this:  
`/opt/logstash/bin/logstash -f /etc/logstash/conf.d/my_config.conf`

i always get an out of memory error (also tried with max. heap of 4G):

> ```
> Settings: Default pipeline workers: 8
> Pipeline main started
> java.lang.OutOfMemoryError: Java heap space
> Dumping heap to /opt/logstash/heapdump.hprof ...
> Unable to create /opt/logstash/heapdump.hprof: File exists
> Pipeline main has been shutdown
> Error: Your application used more memory than the safety cap of 1G.
> Specify -J-Xmx####m to increase it (#### = cap size in MB).
> Specify -w for full OutOfMemoryError stack trace
> 
> ```

My understanding is, that logstash will run once to retrieve the input and then shut down.  
So when i want to retrieve the input in a streaming way, i have to run logstash periodically and use a special query to limit the input?

My logstash config looks like the following:

> input {  
> elasticsearch {  
> hosts =\> ["somehostlocatedsomewhere:9200"]  
> index =\> "_business_"  
> }  
> }

> filter {  
> }

> output {  
> elasticsearch {  
> hosts =\> ["localhost:9200"]  
> sniffing =\> true  
> manage\_template =\> false  
> index =\> "business-%{+YYYY.MM.dd}"  
> document\_type =\> "%{[@metadata][type]}"  
> }  
> }

Is there any way to prevent my out of memory problem?  
I guess, doing it this way logstash tries to retrieve the whole index from system A into its memory and writes it afterwards to disk?

I tried it with a smaller index and this worked with no problem at all.

I appreciate any feedback on my issue also suggestions how to solve my approach.

Regards

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:33am UTC](https://discuss.elastic.co/t/issues-with-logstash-and-elasticsearch-input-plugin-out-of-memory/63717/2 "2017-07-06T04:33:05Z")

</div>