I tested it quite thoroughly with Grok Debugger app first and then even wrote a unit test (with more than 200 different log samples) for that using rspec from logstash and ran that unit test directly on the system where logstash is running. All passed.
But when logstash runs in production grok filter fails for all incoming. I take some of the failing lines and put them into unit test and unit test is passing with them. I'm stuck.
Here is an example log line:
####<Apr 19, 2016 10:57:42 AM CEST> <Info> <Security> <ncby-modd-wbl02> <> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <> <> <> <1461056262679> <BEA-000000> <Disabling the CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true.>
It's logstash 2.3.1.
Logstash is receiving data from filebeat 1.2.3. Could that be the reason?
Filter is actually OK and it works fine when I use file input or stdin input, but the problem seems to be related to some weird combination of filebeat input and grok reg ex., because all messages (including the one above) fail in production with tags added to the message: beats_input_codec_plain_applied, _grokparsefailure. I have other log file formats that go the same path (filebeat -> logstash -> grok -> elasticsearch) and they work, but this particular one does not.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.