Filebeat 5 error when sending to redis

Elastic Stack Beats
1

Getting the following errors when having filebeats send to redis:

Filebeats version: filebeat-5.0.0-alpha5-windows-x86_64
Windows where filebeats is installed: 64 bit win server 2008
Redis version in AWS (Worked with other version - filebeat-1.3.0-windows): redis_version:2.8.24

2016-09-15T17:48:39-07:00 ERR Failed to RPUSH to redis list (write tcp 10.243.196.92:51305->10.218.248.248:6379: i/o timeout) with %!v(MISSING)
2016-09-15T17:48:39-07:00 INFO Error publishing events (retrying): write tcp 10.243.196.92:51305->10.218.248.248:6379: i/o timeout
2016-09-15T17:48:39-07:00 DBG  close connection
2016-09-15T17:48:39-07:00 DBG  send fail

Config file:

filebeat:
  prospectors:
	-
	  paths:
		- "D:/logs/PermissionsServiceJSON.log.node1"
	  input_type: log
	  document_type: permission-service
	  scan_frequency: 1s
	  json.keys_under_root: true
	  json.overwrite_keys: true
  registry_file: "D:/FileBeat/registry"
output:
  redis:
	# Set the host and port where to find Redis.
	hosts: ["abc.xyz.ng.0001.usw2.cache.amazonaws.com:6379"]
	key: "logs"
shipper:
logging:
  level: debug
  # enable file rotation with default configuration
  to_files: true
  files:
	path: D:/FileBeat/filebeat.log

Example of log event from filebeats log:

2016/09/16 00:30:58.824207 client.go:180: DBG  Publish: {
  "@timestamp": "2016-09-13T18:15:16.820Z",
  "@version": 1,
  "beat": {
	"hostname": "a8-3llc-sdks21",
	"name": "a8-3llc-sdks21"
  },
  "class": "org.springframework.core.env.AbstractEnvironment",
  "file": "AbstractEnvironment.java",
  "input_type": "log",
  "level": "DEBUG",
  "line_number": "120",
  "logger_name": "org.springframework.core.env.StandardEnvironment",
  "mdc": {},
  "message": "Initialized StandardEnvironment with PropertySources [systemProperties,systemEnvironment]",
  "method": "\u003cinit\u003e",
  "offset": 1016049,
  "source": "D:/logs/PermissionsServiceJSON.log.node1",
  "source_host": "a8-3llc-sdks21",
  "thread_name": "http-a8-3llc-sdks21/10.243.196.92:8480-1",
  "type": "my_test_log"
}
2

Can you reach the redis-server with the redis-client from your filebeat machine?

3

redis publisher has a default I/O timeout of 5 seconds. Have you tried to increase or disable the timeout?

4

Yes, I can telnet <redis_server> 6379 without any issues.

5

We run all our tests on Redis 3.2 and tested previously on 3.0. Any chance to update the Redis version? Did increasing the timeout help?

6

This topic was automatically closed after 21 days. New replies are no longer allowed.