Dear All,
I am trying to read some custom logs. I am using multi pattern as well. The file is containing only \r or ^M as line terminator. I tried with line terminator configuration auto and carriage_return as well. But the file is not getting read by file beat. Below is my configurations. Appreciated any help or inputs. Attached the snapshot of sample log file as well.
I don't know your context and what other options you could have to get around the issue or to fix the source files so the only option I can suggest is to try a snapshot build of filebeat. If you could change the app generating those file to use LF or CR+LF for newline and make them "compatible" with filebeat, I have to guess you wouldn't be here in the first place...
The line_terminator feature appear merged in master and I think it will be available in the snapshot builds. So you can try that and use that unreleased version of filebeat until a version of filebeat is released with that feature.
That's an imperfect solution I guess, even if it works, but from where I'm standing it's the only helpful suggestion I can make because I can't know what else you could do compared to that. If there exists a better workaround or solution, I'm not in a position to come up with it.
For testing purposes, we generate snapshot builds that you can find here. Please be aware that these are built on top of master and are not meant for production.
By the way I can't suggest a particular build, I don't know what OS, arch, platform, etc you need. If you're not sure which snapshot build to use/try, tell us exactly where you get the current filebeat package you're using and the name of it with all relevant info and we should be able to assist in choosing the right file from that link if you're unsure.
One last thing, if you do try this, the right config according to your description of your log file would have to be:
line_terminator: carriage_return
Since "auto" is the config that is equivalent to the current behavior of filebeat which appear to not work for you because it doesn't support CR only line endings..
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.