Redis input plugin high CPU

Elastic Stack Logstash
1

Hello!

We faced some issues with our Redis & Logstash setup.

syslog_data --> |->rsyslog->redis->logstash->| --> elastic'

RSyslog & Redis & Logstash -- are installed on one physical server (8x cores,). I try to improve and analyze performance of the setup.

In total, we receive ~22K EPS from our firewalls. I disabled all the filters on logstash. So, in the clear environment we have the 22K EPS.

But in this case I saw the CPU usage by redis input thread:

23251 logstash 39 19 23.116g 782196 11232 S 70.0 2.4 5:03.23 [main]<redis
23250 logstash 39 19 23.116g 782196 11232 S 68.3 2.4 5:03.43 [main]<redis
23255 logstash 39 19 23.116g 782196 11232 R 66.3 2.4 5:07.66 [main]<redis
23252 logstash 39 19 23.116g 782196 11232 S 65.3 2.4 5:02.04 [main]<redis
23249 logstash 39 19 23.116g 782196 11232 R 63.0 2.4 5:10.02 [main]<redis
23248 logstash 39 19 23.116g 782196 11232 S 62.0 2.4 5:03.48 [main]<redis
23253 logstash 39 19 23.116g 782196 11232 R 55.3 2.4 5:02.23 [main]<redis
23254 logstash 39 19 23.116g 782196 11232 S 48.3 2.4 4:59.23 [main]<redis

It looks pretty high to me. Our current input config:

input {
redis {
data_type => "list"
key => "rsyslog_logstash"
batch_count => 125
type => syslog
codec => "json"
threads => 8
}
}

Is it normal situation? I'm afraid that cpu usage may impact the performance with enabled filters.

Thanks!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.