Logstash heap size

Elastic Stack Logstash
1

Hello friends!
Just say, I'm not an expert in Logstash or elasticsearch.
I have logstash 1.4.2 cluster, one master and two nodes for data.
I have a lot of logs sending to my cluster every day. And once, logstash stopped to answer for any of my commands, and without thinking twice I reboot the whole system. Things began to happen again and again.
Then I found out a parametr LS_HEAP_SIZE which I increased from 600m to 8g :smile:
But it didnot help as I expected, and the cluster was felt in 5 days after 8 gb was eatten. It looks like OOM killer.
How may I perfom my system?
ES_HEAP_SIZE is 4g for master, and 10g for each of nodes.
My input.conf:

    input {

#################################### Cisco

udp {
    port => 60606
    type => syslog
}

#################################### sysLog

tcp {
    port => 10514
    type => syslog
}

####################################  vmWare

tcp {
    port => 1514
    type => syslog
}

###################################  GELF

gelf {
    codec => "plain"
    host => "0.0.0.0"
    port => 12201
    type => "gelf"
}

################################  ERLANG
udp {
    codec => "json"
    host => "0.0.0.0"
    port => 12211
    type => "erlang"
}

Meanwhile, i found this article: https://github.com/logstash-plugins/logstash-input-lumberjack/issues/10 and think that it's my resolve, but I have only UDP and TCP and GELF inputs :frowning:
Please give me some advice, thanks!

2

Logstash 1.4.2 has a memory leak in the TCP input.

I think it's very disturbing that a 1.4.3 release wasn't released with a fix for this bug. Switch to the most recent Logstash 1.5.0 candidate or build your own 1.4.2 with the small patch that's required.

3

Thank you for answer!
I will study that issue.

4

Today I update my logstash to 1.5.0 and ES to 1.5.2
But LS_HEAP_SIZE still increasing. It is going slower than was at 1.4.2 version but it's increasing.
What i can do? Need I to install ES 1.5.2 to my data nodes?

5

Maybe i wrong, because it's java, and the more I give, the more it eats...

6

Nope, LS HEAP SIZE was increased to 8 gb and my cluster died...
Any advices guys?