# Sending batch to Kafka failed retry loop error

**URL:** <https://discuss.elastic.co/t/sending-batch-to-kafka-failed-retry-loop-error/148871>\
**Category:** Logstash\
**Created:** [September 17, 2018, 6:16pm UTC](https://discuss.elastic.co/t/sending-batch-to-kafka-failed-retry-loop-error/148871 "2018-09-17T18:16:45Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![jamiebuxxx](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jamiebuxxx](https://discuss.elastic.co/u/jamiebuxxx)\
**Post date:** [September 17, 2018, 6:16pm UTC](https://discuss.elastic.co/t/sending-batch-to-kafka-failed-retry-loop-error/148871/1 "2018-09-17T18:16:45Z")

</div>

Hello,

I recently updated our log-brokers to use Logstash 5.6.11. This version is currently using the logstash-output-kafka plugin version 5.1.11. We have 3 log brokers and we recently noticed connections from Kafka dropping from 2/3 of the brokers. We see the following messages in the logs.

```auto
tail -f /var/log/logstash-broker/logstash-plain.log
[2018-09-17T16:59:53,522][INFO][logstash.inputs.udp] Starting UDP listener {:address=>"0.0.0.0:51400"}
[2018-09-17T16:59:53,520][INFO][logstash.inputs.udp] Starting UDP listener {:address=>"0.0.0.0:51401"}
[2018-09-17T16:59:53,529][INFO][logstash.pipeline] Pipeline main started
[2018-09-17T16:59:53,536][INFO][logstash.inputs.gelf] Starting gelf listener (udp) ... {:address=>"0.0.0.0:12201"}
[2018-09-17T16:59:53,547][INFO][logstash.inputs.udp] UDP listener started {:address=>"0.0.0.0:51401", :receive_buffer_bytes=>"106496", :queue_size=>"2000"}
[2018-09-17T16:59:53,550][INFO][logstash.inputs.udp] UDP listener started {:address=>"0.0.0.0:51400", :receive_buffer_bytes=>"106496", :queue_size=>"2000"}
[2018-09-17T16:59:53,551][INFO][logstash.inputs.gelf] Starting gelf listener (udp) ... {:address=>"0.0.0.0:12202"}
[2018-09-17T16:59:53,586][INFO][org.logstash.beats.Server] Starting server on port: 5044
[2018-09-17T16:59:53,575][INFO][org.logstash.beats.Server] Starting server on port: 5045
[2018-09-17T16:59:53,614][INFO][logstash.agent] Successfully started Logstash API endpoint {:port=>9600}
[2018-09-17T17:35:58,665][INFO][logstash.outputs.kafka] Sending batch to Kafka failed. Will retry after a delay. {:batch_size=>1, :failures=>1, :sleep=>0.01}
[2018-09-17T17:35:58,679][INFO][logstash.outputs.kafka] Sending batch to Kafka failed. Will retry after a delay. {:batch_size=>1, :failures=>1, :sleep=>0.01}
[2018-09-17T17:35:58,692][INFO][logstash.outputs.kafka] Sending batch to Kafka failed. Will retry after a delay. {:batch_size=>1, :failures=>1, :sleep=>0.01}

```

The last message just loops until the either the Logstash service is restarted. We have changed nothing from our Kafka cluster or Logstash configs, only the Logstash version.

logstash.conf:

```auto
...
output{
  kafka {
    topic_id => "logstash-broker"
    bootstrap_servers => "logging-kafka-02.domain:9092"
    codec => json
    id => "logging_kafkfa_broker"
  }
}

```

I saw a couple of related issues on the forum which mentioned to make sure the Kafka `server.properties` listeners to PLAINTEXT. We currently have that set already.

kafka server.properties

```auto
listeners=PLAINTEXT://0.0.0.0:9092

```

In the meantime, we are going to have to revert back to the older version of Logstash(5.2) until we are able to resolve this issue. Any help or direction is much appreciated.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 15, 2018, 6:16pm UTC](https://discuss.elastic.co/t/sending-batch-to-kafka-failed-retry-loop-error/148871/2 "2018-10-15T18:16:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.