Filebeat: output to 2 redis servers impossible

Elastic Stack Beats
1

OS: Centos 6.7
Filebeat: filebeat-1.2.3-x86_64.rpm
Configuration:

output:
  redis:
    hosts: ["172.20.0.232:6379", "172.20.0.233:6379"]
    index: logstash

When starting filebeat with debug info, it gives me this:

[root@hostname ~]# /usr/bin/filebeat -c /etc/filebeat/filebeat.yml -e -configtest -d "*"
2016/06/28 08:15:37.634731 beat.go:135: DBG  Initializing output plugins
2016/06/28 08:15:37.634842 geolite.go:24: INFO GeoIP disabled: No paths were set under output.geoip.paths
2016/06/28 08:15:37.634958 redis.go:95: INFO Reconnect Interval set to: 1s
2016/06/28 08:15:37.634987 redis.go:112: INFO [RedisOutput] Using Redis server :0
2016/06/28 08:15:37.635029 redis.go:116: INFO [RedisOutput] Redis connection timeout 5s
2016/06/28 08:15:37.635041 redis.go:117: INFO [RedisOutput] Redis reconnect interval 1s
2016/06/28 08:15:37.635048 redis.go:118: INFO [RedisOutput] Using index pattern logstash
2016/06/28 08:15:37.635056 redis.go:119: INFO [RedisOutput] Topology expires after 15s
2016/06/28 08:15:37.635063 redis.go:120: INFO [RedisOutput] Using db 0 for storing events
2016/06/28 08:15:37.635077 redis.go:121: INFO [RedisOutput] Using db 1 for storing topology
2016/06/28 08:15:37.635087 redis.go:122: INFO [RedisOutput] Using 0 data type
2016/06/28 08:15:37.635862 redis.go:178: WARN Error connecting to Redis (dial tcp :0: getsockopt: connection refused). Retrying in 1s
2016/06/28 08:15:38.637105 redis.go:178: WARN Error connecting to Redis (dial tcp :0: getsockopt: connection refused). Retrying in 1s
2016/06/28 08:15:39.638193 redis.go:178: WARN Error connecting to Redis (dial tcp :0: getsockopt: connection refused). Retrying in 1s

When changing the hosts to a single host, everything works fine

2

It's not recommended to use redis output with filebeat 1.x release, due to data loss and high CPU usage if redis is down. For 5.0 alpha3 release redis output has been completely rewritten getting you features like:

  • proper error handling, data retransmission and back-pressure handling
  • load balancing to multiple redis hosts (for loadbalancing in filebeat consider spooler_size = bulk_max_size * workers to configure lock-step load balancing)
  • SSL support (including support for client authentication) in case redis is run behind some SSL proxy (e.g. stunnel)
  • SOCKS5 proxy support.
3

Thanks, works flawlessly with the new alpha.

Any clue when the alpha will be stable?

4

Not exactly. I assume early fall.

5

This topic was automatically closed after 21 days. New replies are no longer allowed.