Filebeat to kafka failure

Elastic Stack Beats
1

filebeat on server 1 seems to send log data to kafka perfectly, no issues..

set up filebeat on server2 to send log data to kafka and get the below error

2021-04-14T20:46:28.475-0400    INFO    [publisher]     pipeline/retry.go:221   
retryer: send unwait signal to consumer
2021-04-14T20:46:28.475

below is the filebeat.yml

filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

- type: log

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /tmp/filebeat.txt
    #- c:\programdata\elasticsearch\logs\*

  close_inactive: 10m
  ignore_older: 2h
  scan_frequency: 10s
  clean_inactive: 3h
   # Exclude lines. A list of regular expressions to match. It drops the lines that 
are
  
# ================================== Outputs 
===================================

# Configure what output to use when sending the data collected by the beat.

#---- kafka output

output.kafka:
  hosts: ["host1:9092","host2:9092","host3.9092"]
  topic: 'FIT1'
  partition.round_robin:
    reachable_only: true
  required_acks: 1
  compression: gzip
  max_message_bytes: 1000000

added the below after reading some other posts

 close_inactive: 10m
 ignore_older: 2h
 scan_frequency: 10s
 clean_inactive: 3h

still same error.

filebeat version filebeat-7.8.0

2

Did you try to check if Kafka is reachable on the server2? Did you try it with other client/telnet?

3

telnet works, there is network connectivity.

4

tried in debug mode and see this error

Kafka publish failed with: circuit breaker is open

5

this worked after upgrading to filebeat7.12

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.