Hi,
I have a Logstash agent sending lines to a multi broker kafka setup.
Currently I am getting the following error in Logstash log:
[2017-12-27T15:06:15,369][INFO ][logstash.outputs.kafka ] Sending batch to Kafka failed. Will retry after a delay. {:batch_size=>125, :failures=>125, :sleep=>0.01}
[2017-12-27T15:06:15,370][INFO ][logstash.outputs.kafka ] Sending batch to Kafka failed. Will retry after a delay. {:batch_size=>125, :failures=>125, :sleep=>0.01}
[2017-12-27T15:06:15,370][INFO ][logstash.outputs.kafka ] Sending batch to Kafka failed. Will retry after a delay. {:batch_size=>125, :failures=>125, :sleep=>0.01}
[2017-12-27T15:06:15,371][INFO ][logstash.outputs.kafka ] Sending batch to Kafka failed. Will retry after a delay. {:batch_size=>125, :failures=>125, :sleep=>0.01}
Before I set this mutli broker scheme in production I tested on a single node local environment and got the same error. Nevertheless, I overcome this issue by adding the following line to kafka server.properties file:
listeners=PLAINTEXT://192.168.1.70:9092
This solution was taken from:
However in my multi node production setup I do not know how to change this setting.
Should I put only the machine ip on the server.properties file of each broker. Or should I put all broker ips in each server.properties file?
Thanks in advance!