# Logstash is processing logs too slow

**URL:** <https://discuss.elastic.co/t/logstash-is-processing-logs-too-slow/25477>\
**Category:** Logstash\
**Created:** [July 13, 2015, 8:01pm UTC](https://discuss.elastic.co/t/logstash-is-processing-logs-too-slow/25477 "2015-07-13T20:01:20Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![scukonick](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/scukonick/32/44825_2.png) [@scukonick](https://discuss.elastic.co/u/scukonick)\
**Post date:** [July 13, 2015, 8:01pm UTC](https://discuss.elastic.co/t/logstash-is-processing-logs-too-slow/25477/1 "2015-07-13T20:01:20Z")

</div>

Hello,

I have the next configuration.  
ES cluster with 5 servers, each with Intel Xeon 192.99.19.55 and 64GB RAM (3 of the with 128GB).  
Nginx which is load balancing http requests to ES is on one of them.

And I have 3 servers with nginx which are processing about 6000 req/sec each other.  
On each of the servers with nginx is installed Logstash 1.5.2 with the next config:  
[http://pastebin.com/WfHWenh5](http://pastebin.com/WfHWenh5)  
And in the sysconfig I have the next lines changed:

```
LS_HEAP_SIZE="4g"
LS_LOG_FILE=/var/log/logstash/logstash.log
LS_USE_GC_LOGGING="true"
LS_CONF_DIR=/etc/logstash/conf.d
LS_OPEN_FILES=2048
LS_OPTS=""

```

Logstash processes lines well but it does it too slow, like 2 times more slow then new lines appear.  
Servers with logstash have CPU E5-1620 v2 and 64GB RAM.

I tried to change workers count, output worksers, ram settings, etc, but nothing of this help.  
Could you please help me with it?  
I'm sure that elastic and logstash could process this amount of data easily.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 13, 2015, 10:20pm UTC](https://discuss.elastic.co/t/logstash-is-processing-logs-too-slow/25477/2 "2015-07-13T22:20:56Z")

</div>

How slow are they going?

You may also want to increase your flush size to something like 2000, it should help.

---

<div class="post-metadata">

**Author:** ![scukonick](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/scukonick/32/44825_2.png) [@scukonick](https://discuss.elastic.co/u/scukonick)\
**Post date:** [July 15, 2015, 3:31pm UTC](https://discuss.elastic.co/t/logstash-is-processing-logs-too-slow/25477/3 "2015-07-15T15:31:23Z")

</div>

Thank you for you answer!  
As we found out, the problem was not in Logstash but in the ElasticSearch. After some optimizations ElasticSearch works well.

---

<div class="post-metadata">

**Author:** ![Manjunath](https://avatars.discourse-cdn.com/v4/letter/m/c6cbf5/32.png) [@Manjunath](https://discuss.elastic.co/u/Manjunath)\
**Post date:** [August 12, 2015, 3:12am UTC](https://discuss.elastic.co/t/logstash-is-processing-logs-too-slow/25477/4 "2015-08-12T03:12:31Z")

</div>

Hi i am also facing the same issue, can you let me know the optimizations you made in elasticsearch please ?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:32am UTC](https://discuss.elastic.co/t/logstash-is-processing-logs-too-slow/25477/5 "2017-07-06T05:32:13Z")

</div>