filebeat on server 1 seems to send log data to kafka perfectly, no issues..
set up filebeat on server2 to send log data to kafka and get the below error
2021-04-14T20:46:28.475-0400 INFO [publisher] pipeline/retry.go:221 retryer: send unwait signal to consumer 2021-04-14T20:46:28.475
below is the filebeat.yml
filebeat.inputs: # Each - is an input. Most options can be set at the input level, so # you can use different inputs for various configurations. # Below are the input specific configurations. - type: log # Change to true to enable this input configuration. enabled: true # Paths that should be crawled and fetched. Glob based paths. paths: - /tmp/filebeat.txt #- c:\programdata\elasticsearch\logs\* close_inactive: 10m ignore_older: 2h scan_frequency: 10s clean_inactive: 3h # Exclude lines. A list of regular expressions to match. It drops the lines that are # ================================== Outputs =================================== # Configure what output to use when sending the data collected by the beat. #---- kafka output output.kafka: hosts: ["host1:9092","host2:9092","host3.9092"] topic: 'FIT1' partition.round_robin: reachable_only: true required_acks: 1 compression: gzip max_message_bytes: 1000000
added the below after reading some other posts
close_inactive: 10m ignore_older: 2h scan_frequency: 10s clean_inactive: 3h
still same error.
filebeat version filebeat-7.8.0