# Logstash builds Recv-Q for minute long pauses

**URL:** <https://discuss.elastic.co/t/logstash-builds-recv-q-for-minute-long-pauses/38422>\
**Category:** Logstash\
**Created:** [January 5, 2016, 4:25pm UTC](https://discuss.elastic.co/t/logstash-builds-recv-q-for-minute-long-pauses/38422 "2016-01-05T16:25:54Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![ggriffith](https://avatars.discourse-cdn.com/v4/letter/g/e8c25b/32.png) [@ggriffith](https://discuss.elastic.co/u/ggriffith)\
**Post date:** [January 5, 2016, 4:25pm UTC](https://discuss.elastic.co/t/logstash-builds-recv-q-for-minute-long-pauses/38422/1 "2016-01-05T16:25:54Z")

</div>

Hello,

We are sending syslog data over the wire using stunnel to encrypt, and when it gets to the logstash server (on Ubuntu 14.04), we periodically build up a big Recv-Q as the data moves out of the server-side stunnel, and into Logstash's TCP 5000 port.

During this time, the JVM is not logging any garbage collection. In fact, "jstat -gccause" shows numbers remaining static through the duration of this "lockup". Once the Recv-Q drains into logstash and things get moving, we see normal YGC activity which is frequent but keeps things moving along just fine for us.

I wish it was pausing for GC but I can't find any evidence it is, since it seems blocking on the TCP socket coming into logstash.

It appears we run logstash v 1.5.4 using Oracle Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)

/usr/bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -d64 -Dfile.encoding=utf-8 -Dsun.jnu.encoding=utf-8 -XX:PermSize=128m -XX:MaxPermSize=128m -javaagent:/usr/lib/jvm/java-7-oracle/jre/lib/jolokia-jvm-1.2.3-agent.jar=host=localhost,port=8779,policyLocation=file:///usr/lib/jvm/java-7-oracle/jre/lib/jolokia-access.xml -XX:+UseCompressedOops -XX:+AlwaysPreTouch -XX:+ParallelRefProcEnabled -Djava.io.tmpdir=/tmp/logstash -Djava.security.properties=/etc/logstash/java.security -Xmx4096m -Xss2048k -Djffi.boot.library.path=/opt/logstash/vendor/jruby/lib/jni -Xbootclasspath/a:/opt/logstash/vendor/jruby/lib/jruby.jar -classpath : -Djruby.home=/opt/logstash/vendor/jruby -Djruby.lib=/opt/logstash/vendor/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main --1.9 /opt/logstash/lib/bootstrap/environment.rb logstash/runner.rb agent -f /etc/logstash/conf.d -l /srv/log/logstash/logstash.log

Any pointers?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 5, 2016, 8:46pm UTC](https://discuss.elastic.co/t/logstash-builds-recv-q-for-minute-long-pauses/38422/2 "2016-01-05T20:46:57Z")

</div>

What's your config look like?

---

<div class="post-metadata">

**Author:** ![ggriffith](https://avatars.discourse-cdn.com/v4/letter/g/e8c25b/32.png) [@ggriffith](https://discuss.elastic.co/u/ggriffith)\
**Post date:** [January 5, 2016, 9:44pm UTC](https://discuss.elastic.co/t/logstash-builds-recv-q-for-minute-long-pauses/38422/3 "2016-01-05T21:44:47Z")

</div>

Here is the input portion:

input {  
tcp {  
mode =\> "server"  
host =\> "127.0.0.1"  
port =\> 5000  
codec =\> "json"  
}  
}

after that it goes through a few filters then it's off to Graylog:

output {  
gelf {  
host =\> "127.0.0.1"  
port =\> 12201  
}  
}

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 5, 2016, 11:07pm UTC](https://discuss.elastic.co/t/logstash-builds-recv-q-for-minute-long-pauses/38422/4 "2016-01-05T23:07:01Z")

</div>

Are you sure it's not graylog, ie have you changed the output to be a file/stdout and watched the flow?

---

<div class="post-metadata">

**Author:** ![ggriffith](https://avatars.discourse-cdn.com/v4/letter/g/e8c25b/32.png) [@ggriffith](https://discuss.elastic.co/u/ggriffith)\
**Post date:** [January 13, 2016, 4:40pm UTC](https://discuss.elastic.co/t/logstash-builds-recv-q-for-minute-long-pauses/38422/5 "2016-01-13T16:40:02Z")

</div>

The GELF output is UDP. I sniff 127.0.0.1 and see that when Graylog stops receiving messages, nothing is on the line. Blasting UDP shouldn't block, should it?

---

<div class="post-metadata">

**Author:** ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)\
**Post date:** [January 13, 2016, 7:14pm UTC](https://discuss.elastic.co/t/logstash-builds-recv-q-for-minute-long-pauses/38422/6 "2016-01-13T19:14:51Z")

</div>

You're right that blasting TCP shouldn't block. Have you tried checking out the threads in VisualVM ,to see which threads are live and which are idle? A screenshot of that would be very useful

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:15am UTC](https://discuss.elastic.co/t/logstash-builds-recv-q-for-minute-long-pauses/38422/7 "2017-07-06T05:15:51Z")

</div>


