Logstash or redis issue?

Elastic Stack Logstash
1

Recently i have upgraded to ELK latest version. Every thing worked perfectly for some time.
Later logstash shipper syslog events stuck at redis after some time , where as firewall events are flowing normally to logstash indexer.
What would be the possible cause for this situation.

redis input and output configurations looks like below.

Logstash output configuration for redis at shipper
output { redis {
host => [ "host1:7000", "host1:7001", "host2:7002", "host2:7003", "host3:7004", "host3:7005"]
data_type => "list"
key => "syslog" } }
Logstash input configuraton for redis at logstash indexer
input {
redis { host => "host1" port => "7000" data_type => "list" key => "syslog" }
redis { host => "host1" port => "7001" data_type => "list" key => "syslog" }
redis { host => "host2" port => "7002" data_type => "list" key => "syslog" }
redis { host => "host2" port => "7003" data_type => "list" key => "syslog" }
redis { host => "host3" port => "7004" data_type => "list" key => "syslog" }
redis { host => "host3" port => "7005" data_type => "list" key => "syslog" } }

2

That config is really hard to read, can you properly format it?

3

Hi Mark Walkom, I have formatted, please check .

4

The host and post combinations in the input contain duplicates (host2:7001 and host3:7001) and do not match the ones specified in the output. Is this actually the case or just an error when masking the real values?

5

This is copy paste problem by me, i edited the conf file, you can check now. Sorry for inconvenience .

6

Are the firewall events also coming into the Logstash indexer through the Redis queues? Does the indexer have any other inputs defined?

7

Yes firewall events also coming to the logstash indexer (firewall indexer) through redis , but these events flowing.

we are having different config files for syslog and firewall events. inputs defined for syslog config files above.

8

At the same time Redis throwing below warning
Redis connection problem {:exception=>Redis::CommandError: CROSSSLOT Keys in request don't hash to the same slot>, :level=>:warn}