I am introducing the elastic stack into different project in our company.
Unfortunately we are using here some older versions of redhat (5.6), where filebeat is currently not running because the kernel is too old. (facing this issue here:)
I assume that I may find even older operating systems where we would like to get logfiles from.
Beside talking to the collegues about updating these servers, what (free) alternatives do I have to ship logfiles to the stack? We will use kafka or redis as message broker between filebeat and logstash which will ship to elasticsearch, so it would be good to be able to ship the log lines directly to the message broker.
3rd party alternatives are also welcome.
I would just appreciate to have some "plan B" if updating the machines is not possible for some reasons.
Out of curiosity, what specific kernel are you using? In the linked issue it is mentioned that 2.6.18-238.19.1.el5 in RHEL5 works, would you have a chance to try with this kernel, this wouldn't imply a full OS upgrade.
Other alternative could be to try to collect logs with logstash and its file input plugin, this wouldn't be as lightweight as filebeat but it could work. You may need to use an old release and/or an old compatible JVM for that. Take into account that even if this works, this is also not officially supported.
I just encountered kernel 2.6.18-238.el5. Seems to be the default kernel after installation without patches
But since the project is running for a long time yet, I just want to have plan B in hand. Updating or upgrading of course should be the correct way.
A possible workaround is to use rsync to copy the logfiles with a cron job to a server that supports filebeat.
But this works only for one log file type per rsync/server and if you don't need realtime logs (only logs that are not written into can be safly synced without double input - i think - haven't realy tried to make it realtime beacause it was not needed for use)
The problem with this is that rsync sometimes reuses the inode of a deleted logfile during the sync so that filebeat must be configured to forget logs that were already completly read.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.