Grokparsefailure with a pattern verified by grok debug

Elastic Stack Logstash
1

Hello i am using logstash to collect cisco logs like the following one :slight_smile:
2017-08-23T11:03:43.068Z 192.168.1.254 <189>79: *Mar 1 01:17:53.151: %SYS-5-CONFIG_I: Configured from console by console

this is my logstash configuration file:

input {
udp {
port => "8514"
#type => "syslog-cisco"
}

tcp {
port => "8514"
#type => "syslog-cisco"
}
}

filter {
grok {

             match => {
                        "message" => "%{TIMESTAMP_ISO8601:syslog_ng_timestamp} %{IP:original_log_host} (?<seq_no><\d*>\d*): \*(?<C_timestamp>\b\w*\b  \d* \d*:\d*:\d*.\d*): %(?<facility>\b\w*\b)-(?<severity>\d)-%{GREEDYDATA:MNEMONIC}:%{GREEDYDATA:message}"
                      }

     } #grok

}
output {
stdout{
codec => "rubydebug"
}
}

the problem despite the positive response of grokdebug i have always grokparsefailure, i tried trying one pattern at the time to see whats wrong but it didn't work:

1
1.PNG790×505 5.58 KB

2

Show the output from stdout{ codec => "rubydebug" }.

3

this is the output i get :
2

4

And does that message match this grok expression?

%{TIMESTAMP_ISO8601:syslog_ng_timestamp} %{IP:original_log_host} (?<seq_no><\d*>\d*): \*(?<C_timestamp>\b\w*\b  \d* \d*:\d*:\d*.\d*): %(?<facility>\b\w*\b)-(?<severity>\d)-%{GREEDYDATA:MNEMONIC}:%{GREEDYDATA:message}
5

the pattern matches the syslog message but the output does not match at all the semantics .
i deleted %{TIMESTAMP_ISO8601:syslog_ng_timestamp} and the pattern worked but i have an overlap in the message field and there are some other field that took different names :
original_log_host became host
syslog_ng_timestamp became @timestamp

the following image shows the complete output with out %{TIMESTAMP_ISO8601:syslog_ng_timestamp} :

3
3.PNG772×288 24.4 KB

6

Show your complete configuration. Post it as text. Do not post screenshots.

but i have an overlap in the message field

See the grok filter's overwrite option.

7

this is it :

INPUT - Logstash listens on port 8514 for these logs.

input {

udp {

port => "8514"

#type => "syslog-cisco"

}

tcp {

port => "8514"

#type => "syslog-cisco"

}

}

filter {

grok {

	

	 match => { 

	    "message" => "%{TIMESTAMP_ISO8601:syslog_ng_timestamp} %{IP:original_log_host} (?<seq_no><\d*>\d*): \*(?<C_timestamp>\b\w*\b  \d* \d*:\d*:\d*\.\d*): %(?<facility>\b\w*\b)-(?<severity>\d)-%{GREEDYDATA:MNEMONIC}:%{GREEDYDATA:message}"

	     overwrite => [ "message" ]

		  }





 } #grok

}

output {

stdout{

	codec => "rubydebug"

}

}

8 
  "message" => "%{TIMESTAMP_ISO8601:syslog_ng_timestamp} %{IP:original_log_host} (?<seq_no><\d*>\d*): \*(?<C_timestamp>\b\w*\b  \d* \d*:\d*:\d*\.\d*): %(?<facility>\b\w*\b)-(?<severity>\d)-%{GREEDYDATA:MNEMONIC}:%{GREEDYDATA:message}"

But this is the expression that is known to not match?

9

it works well on GROKDebug , did you see some mistakes in it ?

10

it works well on GROKDebug ,

That's irrelevant.

did you see some mistakes in it ?

We're talking in circles. Good luck.