Logstash : High CPU usage and stop logging

yanpeipan · March 16, 2016, 9:35am

my jsstack log: https://gist.github.com/yanpeipan/d9fd90f484d0cd0c9099#file-jsstack-txt
my logstash config: https://gist.github.com/yanpeipan/d9fd90f484d0cd0c9099#file-logstash-nginx-conf

This problem confuse me for a long time. logstash start for few days , it will block self, don't record any log

Christian_Dahlqvist · March 16, 2016, 9:43am

Logstash is generally CPU bound, so seeing high CPU usage when it is busy is expected. Why is this a problem?

yanpeipan · March 17, 2016, 3:25am

logstash start for few days , it will block self, don't record any log

ebuildy · March 17, 2016, 9:20am

How many RAM/CPU Logstash has?

BTW I see you are using Redis, so it's not a problem to "restart" logstash every day if needed (memory leak maybe..).

yanpeipan · March 23, 2016, 5:10am

799.3% CPU usage

ebuildy · March 23, 2016, 8:43am

What about memory?

Java needs memory to avoid GC intensive work (hence CPU consumption).

yanpeipan · March 25, 2016, 10:23am

42% (8058940k total)

this is my /et/init.d/logstash config

name=logstash
pidfile="/var/run/$name.pid"

LS_USER=logstash
LS_GROUP=logstash
LS_HOME=/var/lib/logstash
LS_HEAP_SIZE="1g"
LS_LOG_DIR=/var/log/logstash
LS_LOG_FILE="${LS_LOG_DIR}/$name.log"
LS_CONF_DIR=/etc/logstash/conf.d
LS_OPEN_FILES=16384
LS_NICE=19
LS_OPTS=""