It seems that logstash can send logs to another server that is running elastic search. What is the need for log stash-forwarder? Security?
Setup that I am trying to design.
App Logs Machine Log Server
log stash send to -> elasticSearch
log-stash-forwarder - logstash
logstash-forwarder exists primarily because Logstash has a pretty large deployment overhead (both disk space and RAM) which can be a problem for small machines. Quoting the readme file:
Perceived Problems: Some users view logstash releases as "large" or have a generalized fear of Java.
Actual Problems: Logstash, for right now, runs with a footprint that is not friendly to underprovisioned systems such as EC2 micro instances; on other systems it is fine. This project will exist until that is resolved.
Logstash doesn't seem to have all of the same features to send logs via lumberjack as logstash-forwarder?
Logstash has both input and output plugins for the Lumberjack protocol, so what are you missing, specifically?
I've a different problem / question from the the original questioner but for example the output plugin lacks the ability to set fields (type for example) that the logstash-forwarder can do.
In Logstash fields are added by filters, not output plugins. The
type field is usually set by input plugins (but can be modified by filters). Logstash is a lot more powerful in this regard.
Yup, as we discussed in my thread, I'm all set at least for now