Currently the Beats input supports the Lumberjack v1 protocol, which is documented in the PROTOCOL.md file in the logstash-input-beats repository. I'm currently using this to test the beats input and am interested in expanding it's use in our environment. However, my understanding is Lumberjack has been deprecated for Beats, but I'm not clear if the wire protocol has changed as well. Is this protocol deprecated? How long is it intended to be supported?
Thanks!
beats -> logstash still use lumberjack, but with version number being increased to v2. v2 adds support for a json data-frame and partial ACK that can also be used as keep-alive signal be returning ACK 0 until full batches have been processed.
Okay, so PROTOCOL.md has the JSON frame but still says version 1, is that actually version 2, then?
right, JSON frame is v2.
Okay, so the PROTOCOL.md file is out of date, easily fixed with a PR. So to clarify the original question, the lumberjack protocol is still the official protocol and will be supported long term?
not sure if JSON frame is supported by v1.
beats -> logstash uses lumberjack v2 from the very beginning. This is true for beats 1.x release and upcoming beats 5.x release.
As far as I can tell, the logstash beats input plugin supports both v1 and v2. If and when this will change once logstash forward (which uses lumberjack v1) out of lifetime I don't know. The beats plugin development is mostly driven by logstash. See github
Okay, so Lumberjack v2 is the officially supported protocol as of right now, correct? If so then I think that answers my question.
No idea what you intend by 'officially supported'. Lumberjack v2 is the 'official' protocol used by beats->logstash only. Protocol might change as we see a need to do so to improve beats->logstash integration. There is no interest in 'officially' supporting any other tools using the protocol. Official implementations are available in go-lumber and logstash beat input plugin.
This topic was automatically closed after 21 days. New replies are no longer allowed.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.