Logstash syslog _grokparsefailure errors

[mkain (Manish)]

Hi,

I am using default syslog parser config from elastic site.

input {
  tcp {
    port => 514
    type => syslog
  }
  udp {
    port => 514
    type => syslog
  }
}

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}

output {
  elasticsearch { hosts => ["localhost:9200"] }
  stdout { codec => rubydebug }
}

But with this config many messages fail to parse and I get _grokparsefailure error.

Few eg:

May 14 07:34:44 10.91.116.71 sdwanB2BSlamLog, UAT111, IPC00012, 2, MPLS-WAN, 1, MPLS-WAN, 211, UAT111, 1632, CPE2-MUM, fc15, 0, 2, 1, 7, 7, 0, 0, 0, 0, 0

May 14 07:38:38 10.91.116.71 alarmLog, NV-WC02-N4-MUM, ISP, ipsec-ike-down, 10.5.0.7|14, 1557799309, 1, 0, 2, causeOther, yes, cleared, root, communicationsAlarm, cleared, provider, 45113, "IKE connection with peer 10.5.0.7 (routing-instance ISP-Control-VR) is up", ,

May 14 07:37:01 10.91.116.71 bwMonLog, NV-WC02-N2-BLR, User, 1557800100, 32, 0, 8027, 0, 0, sdwan-acc-ckt-bw-mon-stats, 300036, siteName=NV-WC02-N2-BLR, accCktName=INT-WAN, siteId=24, accCktId=1

How can I fix this ?

Regards,
-Manish

[mkain (Manish)]

Removing pid part from config did the job. Now I dont see parse error anymore.

input {
  tcp {
    port => 514
    type => syslog
  }
  udp {
    port => 514
    type => syslog
  }
}

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program} %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}

output {
  elasticsearch { hosts => ["localhost:9200"] }
  stdout { codec => rubydebug }
}