I can't see a simple way to get data from Filebeat into Flume. Any ideas? Maybe due to Filebeat's youth, there just aren't very many generic ways (HTTP, TCP) to get data out of Filebeat?
Filebeat looks like a rare combination of . . .
Able to read events recorded in log files while Filebeat wasn't running
Compiled or Java
Those qualities made me think it would be a good fit for my go-to log file collector for a new system I'm designing.
The main requirement that seems to eliminate so many log collectors is the ability to "catch up" and read all logged events even if they were recorded while the log collector is offline. Due to audit requirements, this system can't miss events due to network connection problems or any other reason; all logged events must be persisted in long-term storage. I'm trying to find a log collector that will do so and therefore avoid having to monkey with how each and every system, app, OS manages log file rotation, etc.