Hi! I'd like to have Filebeat set up the way that it would consume messages from Kafka topic and then send them to Elasticsearch. Is there a way, how to set it all up in docker-compose? I could register Logstash as a Kafka consumer so I assume that it should be also possible for Filebeat. In the log, I can see that it loaded the config file. I don't see it, however, in the list of Kafka's consumers. So Filebeat doesn't consume any messages, even though they are in Kafka. Can somebody point me in the right direction? I assume that it should be a really basic configuration. Yet, I'm missing something.
The FIlebeat part of docker-compose.yml:
the content of the filebeat.yml file:
- type: kafka