I want to save all _grokparsefailure entries in a file to review them later, however

Elastic Stack Logstash
1

My output configuration portion:

if "_grokparsefailure" in [tags] { file { path => "/home/myaccount/testfolder/grokfailures" file_mode => 0600 } }
  else
   {
    file { path => "/home/myaccount/testfolder/groksuccess" file_mode => 0600 }
   }

My grok filter works as a charm and all logs go into groksuccess.
However, when I deliberately change the grok filter not to match my entries I can see them in /var/log/logstash/logstash-plain.log being tagged with _grokparsefailure and yet they don't go into my grokfailures file.

Any help please?

2

I think the file plugin can only be used in the output object. So you would need to do it like this:

output {
    if "_grokparsefailure" in [tags] {
        file { 
            path => "/tmp/grok-failed.log"
        }      
    }
3

Indeed my code snipped is from the output object and still not working. Very odd. Thank you for your reply anyway.

4

Does that file exist, and do you have permissions to write to it?

Try removing the file_mode option, it does not work the way most folks would expect.

5

Hi @Badger, yes the file didn't existed and I created it. I also removed the file_mode and again no joy. Again, as I said the funny thing is that file groksuccess receives all the logs but whey they are tagged with _grokparsefailure they just won't go into file grokfailures
This is a single entry when I alter the regex to fail to match the logs and as you can see the _grokparsefailure tag is in there

image
image1896×74 43.7 KB

6

Anyone else? Still haven't found a solution and its a bit frustrating.
Thank you all in advanced.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.