I have problems to communicate with my broker as output.
My actually working configuration is :
Filebeat -> Logstash 1 -> RabbitMQ -> Logstash 2 -> ElasticSearch
Logstash1 is only an intermediate between FB and the broker (since Filebeat doesn't support RabbitMQ as output)
Logstash2 is the worker which transform the data
Now , i tried to remove Logstash1 to use only one broker :
Filebeat -> Kafka -> Logstash -> ElasticSearch
But it doesn't seems to work , her is my filebeat.yml
filebeat: prospectors: - paths: - C://filebeat-1.1.1-windows//logs//*.log* input_type: log output: kafka: host: "localhost:9092" topic: "test" use_type: true client_id: "beats" worker: 1 max_retries: 0 shipper: logging: files: rotateeverybytes: 10485760 # = 1MB
My message error :
Error Initialising publiser: No outputs are defined. Please define one under the output section.
As i said it works with logtsash or console as output.
My configuration :
Filebeat : filebeat-1.1.1-windows
Kafka : kafka_2.11-0.9.0.1
Java version : 1.7.0_45
Windows : 8.1Entreprise 64bits
Thanks for helping