Doese a 'local' filebeat installation makes sense?


(Patrick) #1

hi, i got a windows server on which i have a network share mounted with several logfiles I want to send to logstash. logstash (and actualy the whole elasticstack) is installed on that same server. I can use file input in logstash to get the log data.
But my question is: do i have any advantage if I would install filebeat on the same machine and would use the beats protocol to ship the log data to logstash instead of using file input?


(Magnus Bäck) #2

I don't think there would be a technical advantage if you're going to run Logstash on that machine anyway. However, if you have other machines where you'd run similarly configured Filebeat instances it might be easier from a management perspective to have the same configuration everywhere.


(Patrick) #3

okay alright. i keep that in mind if some servers will added as shippes. thank you for the quick respond @magnusbaeck :slight_smile:


(ruflin) #4

I think @magnusbaeck mentioned the most important point. If you use Filebeat it allows you to add machines without changing your setup.


(Patrick) #5

thank you @magnusbaeck and @ruflin.

i got my setup running with the file input but i got a problem with a specific log file -logstash dosent parse the data in it. this file will be generated every friday and will gain up to more than 50 MB during the week (not so much logevents entries in the files but long lines, i cut the longer lines off with a mutate filter and message will be dropped, too cause the information in the longer lines are useless for me). does logstash have problemes with larger files? would it better to try this with a local filebeat?

in addition to the file size the application which writes the logfile adds after every entry a new empty line (the next log event will be written in this line so no spaces between log events in the log just temporyry new lines). i have no write permission on the file - so i copied the content of the file in a new one and replaced the empty line at the end and startet logstash with the filepath of this new file and it parsed every entry correctly... do you have an idea to solve this problem with the file from the server?


(Magnus Bäck) #6

I'm not aware of Logstash having issues with line length, but there probably is a limit somewhere. How long lines are we talking about here?

in addition to the file size the application which writes the logfile adds after every entry a new empty line (the next log event will be written in this line so no spaces between log events in the log just temporyry new lines).

Do you mean that the file always ends with a newline character? That's exactly what Logstash expects. If not I don't understand what you mean by "temporary new line".


(system) #7