Multiple match option in grok

Elastic Stack Logstash
1

I have a question about several matches in grok filter.
Am I right to assume that with multiple matches, even if the log will be parsed successfully by one of them, it will still be _grokparsefailure in tags, cause of mismatch with the other match expressions?

2

To avoid misunderstandings, please use examples as much as possible instead of describing things in words. Are you talking about a single grok filter with multiple expressions?

grok {
  match => {
    "message" => ["expr1", "expr2", ..., "exprN"]
  }
}

In this case you'll only get a _grokparsefailure if none of the expressions matched.

3

Oh, thank you for the answer! That's what I meant, sorry for being too vague.

4

For some reason, though, code like this:

filter {
    if [type] == "syslog" {
        grok {
            match => {
                "message" => ["%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: Launching batch job %{NUMBER:job_id} for UID %{POSINT:slurm_user_id}","%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: launch task %{NUMBER:job_id} request from %{PROG:slurm_host} (\(port %{POSINT:slurm_port}\))","%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: _run_prolog: run job script took usec=%{NUMBER:slurm_usec}","%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: _run_prolog: prolog with lock for job %{POSINT:job_id} ran for %{NUMBER:slurm_time}"]
       }
       add_field => [ "received_at", "%{@timestamp}" ]
       add_field => [ "received_from", "%{host}" ]
       }
   }

   syslog_pri { }

   if !("_grokparsefailure" in [tags]) {
        mutate {
           add_tag => "parsed_by_slurm_filter"
        }
    }
    mutate {
        rename => { "syslog_message" => "message" }
    }
}

gets logstash to stop working. Why?

5

What does "stop working" mean, exactly?

6

That means that logstash doesn't compile, and therefore it is unable to send or parse logs in any way.

7

Seems to work fine here:

$ cat test.config 
filter {
    if [type] == "syslog" {
        grok {
            match => {
                "message" => ["%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: Launching batch job %{NUMBER:job_id} for UID %{POSINT:slurm_user_id}","%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: launch task %{NUMBER:job_id} request from %{PROG:slurm_host} (\(port %{POSINT:slurm_port}\))","%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: _run_prolog: run job script took usec=%{NUMBER:slurm_usec}","%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: _run_prolog: prolog with lock for job %{POSINT:job_id} ran for %{NUMBER:slurm_time}"]
       }
       add_field => [ "received_at", "%{@timestamp}" ]
       add_field => [ "received_from", "%{host}" ]
       }
   }

   syslog_pri { }

   if !("_grokparsefailure" in [tags]) {
        mutate {
           add_tag => "parsed_by_slurm_filter"
        }
    }
    mutate {
        rename => { "syslog_message" => "message" }
    }
}
$ /opt/logstash/bin/logstash -f test.config --configtest
Configuration OK
8

Yeah, everything works now, thanks. It was another software that stood in the way.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.