Logstash with redis cluster problem


(Liuzg459) #1

Hi there! I want to use redis cluster instant of one instance. here is my framework:
logstash shipper ->redis cluster -> logstash indexer -> elasticsearch -> kibana

Here is my redis config:

[root@localhost src]# ./redis-trib.rb info 192.168.1.153:4000
192.168.1.153:4000 (4c017302...) -> 0 keys | 5461 slots | 0 slaves.
192.168.1.153:4002 (12b14519...) -> 1 keys | 5461 slots | 0 slaves.
192.168.1.153:4001 (59ca7fa9...) -> 0 keys | 5462 slots | 0 slaves.
[OK] 1 keys in 3 masters.
0.00 keys per slot on average.
[root@localhost src]# redis-cli -c -p 4000
127.0.0.1:4000> set "test" "test"
-> Redirected to slot [6918] located at 192.168.1.153:4001
OK
192.168.1.153:4001> set "abc" "abc"
OK
192.168.1.153:4001> set "hello" "world"
-> Redirected to slot [866] located at 192.168.1.153:4000
OK
192.168.1.153:4000>

Here is my logstash shipper config:

input{
log4j {
port => 4561
type => "log4j-json-test"
}
}
output{
redis {
data_type => "list"
host => "192.168.1.153:4000"
key => "logstash_test"
}
}

Here is my logstash indexer config:

input{
redis{
host => "192.168.1.153"
port => 4000
key => "logstash_test"
data_type => "list"
}
}
output{
elasticsearch {
hosts => "192.168.1.153"
}
}

Here is the logstash shipper error log:

[jason@localhost logstash]$ ./logstash-2.1.1/bin/logstash agent -f conf/client1.conf
Settings: Default filter workers: 2
Logstash startup completed
Failed to send event to Redis {:event=>#<LogStash::Event:0x1c747a72 @metadata={}, ...................

Here is the logstash indexer error log(I think the resason is not the same with shipper error):

Redis connection problem {:exception=>#<Redis::CommandError: CROSSSLOT Keys in request don't hash to the same slot>, :level=>:warn}


(Mark Walkom) #2

I'd put the port number in quotes.


(Liuzg459) #3

Negtive,I tried,but it make no differents


(Mark Walkom) #4

Can you telnet from the indexer to redis?


(Liuzg459) #5

Ofcourse,they are on the same server.

I suspect that the configuration was not correct.


(system) #6