I have a log which cannot directly extract using grok patterns under logstash filter. I have tested full grok pattern for the log in grok debugger and it works fine. But when I run the grok pattern under logstash filter gives grok failure error.
So I need to break full message into small parts and apply the grok individually.
I need to know, whether above mention requirement is possible under logstash ?
Do you know why the log is being marked as grokparse failure? Maybe we could help you solve that issue.
If you are dead set on splitting up your log, how are you reading the log in? Through a file? Syslog? If you are reading a the log in using the file input that has a configurable option called delimiter which allows you to set a new line delimiter.
Perhaps you could attach a sample log, the grok you constructed and your logstash.conf file, removing any sensitive information.
and here are the sample log. Some sensitive information are masked here.
<46>Oct 10 11:44:29 XXX-XX-XX XXXXS: [135:2:2] internal: session established [Impact: Potentially Vulnerable] From "XXX-XX-XX" at Tue Oct 10 11:44:29 2017 UTC [Classification: A TCP Connection was Detected] [Priority: 3] {tcp} 192.168.1.1:23 (unknown)->10.10.10.10:XXXXX (unknown)
<46>Oct 10 11:44:32 XXX-XX-XX XXXXS: [119:4:1] http_inspect: BARE BYTE UNICODE ENCODING [Impact: Potentially Vulnerable] From "XXX-XX-XX" at Tue Oct 10 11:44:31 2017 UTC [Classification: Not Suspicious Traffic] [Priority: 3] {tcp} 10.10.10.XXX:XXXXX (unknown)->192.168.1.1:80 (unknown)"
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.