For those using logstash-forwarder but also using Hortonworks stack (HDP and HDF)
I have baked an experimental listener that allows Hortonworks Dataflow (a.k.a. Apache NiFi) to accept connections from logstash-forwarder agents.
The rationale for that is to allow for a drop in replacement for simpler pipelines (e.g. logstash-forwarder -> pipeline -> HDFS / Kafka / whatever) while providing stronger delivery guarantees than Logstash currently offers (NiFi's documentation provides insight over its data provenance and guarantees).
The code is still experimental and I would strongly appreciate if people could assist by testing it against a more diverse set of environments.
Yes, I am aware of the deprecation, however its widespread use and the fact other applications make use of the protocol (e.g. node-lumberjack, python beaver, etc) means its use will continue for a while.
I plan to introduce support for libbeats once the protocol is formalised as its current implementation differs from its documentation
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.