LS output failed

I'm seeing the following INFO message, the connection to Kafka is lost and I'm unable to connect to it.

2018-03-14T14:13:02,743][INFO ][logstash.outputs.kafka ] Sending batch to Kafka failed. Will retry after a delay. {:batch_size=>1, :failures=>1, :sleep=>0.01}

I've validate that Kafka is up and running.


this line already exist in Kafka. I could telnet to the Kafka server even though LS is unable to output to it.

ES 5.6.8

any help would be great?

ran a debug, but got way too many logs. I noticed that it failed at exactly 19:00 three days in a row.

Does the connection ever resume? Does it require a restart of Logstash?

The timing is suspicious; could a scheduled task be restarting the Kafka process? In a previous job, we had a finnicky Kafka installation that someone "fixed" by killing it daily in cron.

It doesn’t reconnect. I have to restart Logstash to get the connection going.

I found something interesting.

If I reboot the box, it will work, but a few hours later, the error is back. But if I restart it using "initctl" it will work fine for a while. What the different in the two starting method?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.