Kafka Output Recurring I/O Timeout

Elastic Stack Beats
1

Hi,

We're seeing Filebeat getting:

	kafka/log.go:53	producer/broker/1036 state change to [closing] because write tcp 10.200.1.158:49334->10.200.3.121:9092: i/o timeout

This causes Filebeat to re-connect to Kafka brokers and reduce speed.
Anyone with a solution or a way to debug?

2

Hi @Or_Arnon,

Is any event successfully sent or all connections to Kafka fail?

3

All events are sent. However, we do get these timeouts which forces Filebeat to re-discover all partitions.

4

this is our complete config:

filebeat.inputs:

  • type: log
    enabled: true
    paths:
    • /ephemeral/log/haproxy/haproxy*
      exclude_lines: ['^C']
      fields_under_root: true
      json.keys_under_root: true
      json.add_error_key: true
      json.message_key: message
      json.ignore_decoding_error: true
      scan_frequency: 1s
      multiline.pattern: '^{'
      multiline.match: after

filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: true

filebeat.config.inputs:
reload.enabled: true
reload.period: 60s

output.kafka:
enabled: true
hosts: [ .... ]
worker: 1
version: 0.11.0.0
compression: snappy
bulk_max_size: 4098
required_acks: 0
message.max.bytes: 20000000
client_id: "gateway_haproxy"
topic: "test_filebeat"
partition.round_robin:
group_events: 1
reachable_only: true

logging.level: info

http.enabled: true

xpack.monitoring:
enabled: true
elasticsearch:
hosts: [ ... ]

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.