# Memory issue with logstash s3 input **URL:** \ **Category:** Elasticsearch\ **Created:** [May 5, 2015, 12:30pm UTC](https://discuss.elastic.co/t/memory-issue-with-logstash-s3-input/23559 "2015-05-05T12:30:55Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![vangap](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vangap/32/44927_2.png) [@vangap](https://discuss.elastic.co/u/vangap)\ **Post date:** [May 5, 2015, 12:30pm UTC](https://discuss.elastic.co/t/memory-issue-with-logstash-s3-input/23559/1 "2015-05-05T12:30:55Z")
I have Elasticsearch, Logstash running on a m3.xlarge. ES 1.5.2 Logstash 1.5.0-rc3 ArchLinux I am trying to index 3 months log data into ELK, but after some time I see that kernel is killing the processes because of an OOM error. [http://pastie.org/10155132](http://pastie.org/10155132) I looked at the memory metrics, which was constant around 70% (with 50% mlocked by ES). After few minutes of starting logstash, OOM kills ES. I noticed that /tmp/logstash size is 5.5 GB. logstash ran fine for some time when I first started indexing. Is logstash trying to bulk index all these files? How do I avoid this? can I throttles this some how? If I change this temporary location, will that be of any help? Thanks. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f25c3ec4-8343-4bc9-ba73-e37eb16413f7%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f25c3ec4-8343-4bc9-ba73-e37eb16413f7%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [May 5, 2015, 12:37pm UTC](https://discuss.elastic.co/t/memory-issue-with-logstash-s3-input/23559/2 "2015-05-05T12:37:43Z")
On Tuesday, May 05, 2015 at 14:30 CEST, Pradeep Reddy [pradeepreddy.manu.iitkgp@gmail.com](mailto:pradeepreddy.manu.iitkgp@gmail.com) wrote: > I have Elasticsearch, Logstash running on a m3.xlarge. > ES 1.5.2 > Logstash 1.5.0-rc3 > ArchLinux > I am trying to index 3 months log data into ELK, but after some time I > see that kernel is killing the processes because of an OOM error. > [http://pastie.org/10155132](http://pastie.org/10155132) > I looked at the memory metrics, which was constant around 70% (with > 50% mlocked by ES). > After few minutes of starting logstash, OOM kills ES. Please ask Logstash question on the logstash-users list. Or, even better, use the recently announced discussion forum at [https://discuss.elastic.co](https://discuss.elastic.co). See [https://groups.google.com/d/topic/elasticsearch/IsYD2ScoE\_Y/discussion](https://groups.google.com/d/topic/elasticsearch/IsYD2ScoE_Y/discussion). [...] -- Magnus Bäck | Software Engineer, Development Tools [magnus.back@sonymobile.com](mailto:magnus.back@sonymobile.com) | Sony Mobile Communications -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/20150505123742.GA32018%40seldlx20533.corpusers.net](https://groups.google.com/d/msgid/elasticsearch/20150505123742.GA32018%40seldlx20533.corpusers.net). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![vangap](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vangap/32/44927_2.png) [@vangap](https://discuss.elastic.co/u/vangap)\ **Post date:** [May 5, 2015, 12:53pm UTC](https://discuss.elastic.co/t/memory-issue-with-logstash-s3-input/23559/3 "2015-05-05T12:53:55Z")
Thanks, I didnt know that there is a separate group. Will post there. On Tuesday, May 5, 2015 at 6:00:55 PM UTC+5:30, Pradeep Reddy wrote: > I have Elasticsearch, Logstash running on a m3.xlarge. > ES 1.5.2 > Logstash 1.5.0-rc3 > ArchLinux > > I am trying to index 3 months log data into ELK, but after some time I see > that kernel is killing the processes because of an OOM error. > [http://pastie.org/10155132](http://pastie.org/10155132) > I looked at the memory metrics, which was constant around 70% (with 50% > mlocked by ES). > > After few minutes of starting logstash, OOM kills ES. > > I noticed that /tmp/logstash size is 5.5 GB. logstash ran fine for some > time when I first started indexing. > Is logstash trying to bulk index all these files? > > How do I avoid this? can I throttles this some how? If I change this > temporary location, will that be of any help? > > Thanks. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f91b0fbd-90d0-4220-9dba-008860d6c36a%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f91b0fbd-90d0-4220-9dba-008860d6c36a%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:15am UTC](https://discuss.elastic.co/t/memory-issue-with-logstash-s3-input/23559/4 "2017-07-06T00:15:39Z")