I need help with what I thought was a very simple issue, but turns out it isn't. I have updated from logstash 7.x to 8.x, and even though all of my tests have passed correctly, I found several issues with one of the pipelines that integrates firewall logs using a modified version of PFelk. It is more or less working now, but I still don't manage to receive logs from the "haproxy" side. I believe the problem is due to some new lines in the message. I don't what's changed and why they are a problem now, but I can't get rid of them. I have tried all the solutions proposed in similar posts, but none work.
In my tests, this type of input is discarded, not even recognized by logstash. In production it simply fails with a _grokparsefailure. I tried adding the gsub filter to remove "\n", but it doesn't do anything unless I escape the "\n" in the original message.
Turns out it was all a very funny mistake. I had the version pinned, so it did not upgrade to 8.X; however, my pipelines were changed to fit 8.X and changes made to pfelk... So yeah, that went very wrong... After I noticed and upgraded, everything slowly fell into place.
Just as a future confirmation, the mutate to remove "new lines" did work.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.