I have a log file and I am parsing it through Logstash and storing it in some place. But the the problem is that some lines in the log file do not always match my grok pattern and are therefore tagged as 'grokparsefailure' etc automatically. I do not tag any line explicitly, so the lines which are automatically tagged by logstash are the wrongly structured lines and I want these lines to be skipped i.e. I don't want these output lines to appear in the output at all (lines which have tags).
This construct is present in many examples online, but what if I still want the line logged, just differently?
If my grok-filter failed, I do not want any other filters applied, but still want to record the entire message in a separate output -- how would I achieve that?
The separate output part is easy -- by using checking for the [tags] -- but what about avoiding all the other filters without dropping the event? Must I move them all into a condition checking for _grokparsefailurenot being among the [tags], or is there some other way? Thank you!
@magnusbaeck this method works for me too, but I want to know why the line with _grokparsefailure line are still sent to ES. I am new to use logstash and ES, from my perspective, the filter is to filter some lines which will be grok-failed and not sent to ES, is there something wrong, thanks a lot.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
