Difference between File Beats and Logstash file input

Elastic Stack Logstash
1

Hi,
I have all my log files accessible on the machine where Logstash is running and have been using the file input configuration of Logstash. Is there any functional difference is using this approach vs running File Beats on the same machine? Do the two share the same source code for reading from log files? What is the preferred approach?

Thanks,
Sanjiv

2

I have all my log files accessible on the machine where Logstash is running and have been using the file input configuration of Logstash. Is there any functional difference is using this approach vs running File Beats on the same machine?

Not really.

Do the two share the same source code for reading from log files?

No.

What is the preferred approach?

Since the implementation is different there might be slightly different behavior in corner cases. Apart from that it's mostly up to what makes the most sense for you. If I use Filebeat for collecting a particular kind of log file on all servers I'd use Filebeat everywhere instead of making an exception for the Logstash server(s) which theoretically wouldn't have needed Filebeat.

The file input and Filebeat have slightly different tuning options too.

3

Thanks for clarifying. I have setup all our client and server logging to occur on a shared network path with daily log4j rolling file appenders and so all the logs are accessible from the machine running Logstash. Since we had the flexibility to log to a shared network path, the setup seemed a lot simpler than running and managing file beats on each machine.

The machine running LS is on Windows and with LS 2.2.2 I was having some issues where overnight when the log file rolled, LS wasn't processing changes to the new log files. I read various posts about this being a potential issue due to the windows mount caching and LS possibly not detecting the changes. This was explicitly noted in the release notes of an older version of LS but in the few days of upgrading to LS 5.0, I no longer see this behavior and the rotated log files are being read as expected so I'm hoping this was an issue that was addressed with the upgrade to v5.0.

I suppose if I was still having issues with file input of LS 5.0 not detecting rolled files I would try running FileBeats on the machine running Logstash to see if it is able to work better with windows network shares but for simplicity of having a single process I think I will stick with using the LS file input for now.

Please do let me know if you're aware of any subtleties in FileBeats vs file input with regards to reading files from windows shared drives.

Thanks,
Sanjiv