Webhdfs output in Filebeat

Elastic Stack Beats
1

Hi,

are you open to the idea of webhdfs output for Filebeat?
Something like https://www.elastic.co/guide/en/logstash/current/plugins-outputs-webhdfs.html for Logstash.

I am willing to work on that but I want to know if it will be merged.
I see that Filebeat has only ELK stack outputs so far and Logstash has many others, but we would love to use Filebeat because of its performace.

Vit

2

Hi @tivvit

Best have a look at the conversation in this thread and the last comment: https://github.com/elastic/beats/issues/581#issuecomment-345084702 The same applies to webhdfs.

For your setup I would suggest to have LS in the middle to forward events to wbhdfs and have Filebeat on the edge nodes sending data to LS if that is an option.

BTW: There is also always the path with the plugin but it's a bit more complex.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.