Whether the filebeat enabled mysql module supports direct pass to kafka


(tycho_yang) #1

I want to use the filebeat mysql module to directly pass the stream to the kafka cluster through the own pipeline, but found that the delivery is unsuccessful, the kafka cluster consumer does not generate logs, I try to pass directly to es is normal. So I want to confirm whether the mysql module is passed directly to kafka or not.


(tycho_yang) #2

Did you know that BUG or did not support it?
filebeat log


(Mario Castro) #3

Hi @ytc301

It will be useful if you can provide the version of Filebeat, Kafka and your configuration file. Maybe your output is not configured properly.

Thanks


(tycho_yang) #4

Filebeat configuration should be correct, my kafka output can be output directly to es can be normal, and mysql slow is normal serialization, then the output of the ES is closed on the configuration of kafka output, kafka consumer has no data produce. In addition, Kafka is normal, because other businesses are also using that kafka cluster, and I have already created the corresponding topic in the kafka cluster in advance.

filebeat version: 6.5.4
kafka version: kafka_2.11-0.10.1.0


(tycho_yang) #5

Is there any progress?


(tycho_yang) #6

What's the problem?


(tycho_yang) #7

@Mario_Castro ????