File-Beat-Kafka-Producer-config

Elastic Stack Beats
1

Hi All,

I am using kafka as Filebeat output .

I am seeing below exception while sending data to kafka topic ,

2018-07-09T13:54:10+02:00 INFO producer/broker/[[0]] maximum request accumulated, waiting for space

2018-07-09T13:54:14+02:00 INFO producer/broker/[[0]] maximum request accumulated, waiting for space

2018-07-09T13:54:17+02:00 INFO producer/broker/[[1]] maximum request accumulated, waiting for space

2018-07-09T13:54:18+02:00 INFO producer/broker/[[0]] maximum request accumulated, waiting for space

2018-07-09T13:54:18+02:00 INFO producer/broker/[[1]] maximum request accumulated, waiting for space

2018-07-09T13:54:18+02:00 INFO producer/broker/[[0]] maximum request accumulated, waiting for space

2018-07-09T13:54:19+02:00 INFO producer/broker/[[2]] maximum request accumulated, waiting for space

2018-07-09T13:54:20+02:00 INFO producer/broker/[[1]] maximum request accumulated, waiting for space

2018-07-09T13:54:21+02:00 INFO producer/broker/[[2]] maximum request accumulated, waiting for space

2018-07-09T13:54:22+02:00 INFO producer/broker/[[2]] maximum request accumulated, waiting for space

2018-07-09T13:54:22+02:00 INFO producer/broker/[[2]] maximum request accumulated, waiting for space

2018-07-09T13:54:25+02:00 INFO producer/broker/[[0]] maximum request accumulated, waiting for space

2018-07-09T13:54:25+02:00 INFO producer/broker/[[0]] maximum request accumulated, waiting for space

2018-07-09T13:54:29+02:00 INFO producer/broker/[[2]] maximum request accumulated, waiting for space

2018-07-09T13:54:30+02:00 INFO producer/broker/[[2]] maximum request accumulated, waiting for space

2018-07-09T13:54:30+02:00 INFO producer/broker/[[1]] maximum request accumulated, waiting for space

2018-07-09T13:54:30+02:00 INFO producer/broker/[[1]] maximum request accumulated, waiting for space

2018-07-09T13:54:30+02:00 INFO producer/broker/[[2]] maximum request accumulated, waiting for space

I have 3 broker , and my topic has 3 replication factor and 3 partition . .

Please help me in resolving the same as looks like i have to update something in my producer/broker config .

Thanks

2

Hi @SJN8,

This is not an exception but an info message. kafka client's buffer is full, and it's probably waiting for some ACKs before sending more messages.

Best regards

3

Yes , This is resolved by updating socket config .

Thanks .

4

Hi, my program has the same problem, how do you solve it?

5

I have solved by increasing Buffer memory and producer config of Filebeat and Kafka

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.