The elastic doc https://www.elastic.co/guide/en/logstash/current/plugins-inputs-lumberjack.html states " Consider using the Beats input plugin instead. The Beats input implements the Lumberjack protocol v1 and v2."
However, there is insufficient information about how to make such a consideration and I am looking here to surface , from the community, some of the pros/cons of using lumberjack input vs beats input in logstash.
Today, I have a logstash pipeline with beats input and another with lumberjack input.
- We have beats output to the beats intake of logstash2 and integration tests ok.
- We have client logstash1 output to lumberjack intake on logstash2 ok but no integration tests yet.
Here are some approaches to not only adding a test for the logstash to logstash but changing the production to not directly even use a lumberjack intake.
2a. Introduce a client filebeat, for testing, output to logstash2 using the existing lumberjack intake. This means I don't have to stand up a client logstash to test the lumberjack intake. I tried this and it is not viable. I found the event delivered to lumberjack intake must have certain fields like "line","file","offset" set to certain values of expected type. It's a waste of time to essentially develop a logstash-forwarder with a beats configuration.
2b. Introduce a client logstash1, for testing, outputing via lumberjack to the the existing lumberjack intake of logstash2. This means I have to stand up a client logstash and I know it will work.
2c. Introduce a client logstash1, for testing, outputing via lumberjack to the the existing beats intake of logstash2. I am reluctant to do this if I have a known working lumberack intake; I see no benefits.
2d. Remove the logstash2 lumberjack intake from logstash2. The logstash1 is outputing lumberjack to logstash2 beats intake. For testing, stand up a logstash1 to exercise this.
2e. I know there is a possibility to even not even support logstash to logstash but go strictly with beats to logstash which might even be ideal but it impacts our application teams so I will defer that alternative for now.
Thoughts?