Grokparsefailure syslog which has no pid

[Bhargav_Narine]

Hi,

I have written a plugin to resolve host names using DNS lookup.

It was working fine and recently, dns is been blocked in lab. And, we have updated /etc/hosts for resolving it locally. Still, _grokparse failure is happening for DNS. Is this expected behavior? Can you please help?

Filter

if [type] == "syslog" {
mutate {
add_field => { "clustername" => "%{host}" }
}
dns {
action => "replace"
reverse => [ "clustername" ]
add_tag => [ "dns_lookup" ]
}
}

Log

[2017-06-06T05:19:38,369][DEBUG][logstash.pipeline ] output received {"event"=>{"@timestamp"=>2017-06-06T05:19:38.361Z, "port"=>54923, "clustername"=>"unclecluster", "@version"=>"1", "host"=>"10.58.205.45", "message"=>"Jun 6 11:13:58.413639 info CLA-1 dhcpd: DHCPACK on 169.254.1.62 to 00:a0:a5:83:22:37 via bond0", "type"=>"syslog", "tags"=>["dns_lookup", "_grokparsefailure"]}}

Here clustername is actually resolved but still grokparsefailure comes.

[Bhargav_Narine]

Hi,

Modified the subject accordingly
After further investigation, it looks like issue is with grok parsing of syslog. My grok is as follows

grok {
match => { "message" => [ "%{SYSLOGTIMESTAMP:syslog_timestamp} %{WORD:severity} %{NOTSPACE:node} %{PROG:process}(?:[%{POSINT:pid}]): %{GREEDYDATA:logmsg}" ] }
}

In my syslog entry, I dont get pid for all. Only for few, I get pid and for remaining it only process name. So, is there any way of parsing both? Thanks.

[magnusbaeck (Magnus Bäck)]

Why make up your own grok expression for syslog messages when Logstash ships with syslog patterns that you can use and that don't have this problem?

https://www.elastic.co/guide/en/logstash/current/config-examples.html#_processing_syslog_messages