Parsing FreeBSD syslog

Hi,

I am trying to parse FreeBSD syslog. The syslogs are sent to logstash via filebeat. I currently use this grok:

filter {
if [type] == "syslog" {

grok {
  match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
  add_field => [ "received_at", "%{@timestamp}" ]
  add_field => [ "received_from", "%{host}" ]
}

syslog_pri { }

date {
  match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
}

}
}

and the output of this is:

{
"_index": "filebeat-2017.08.17",
"_type": "syslog",
"_id": "AV3wahQdJ57xNwGFxihH",
"_version": 1,
"_score": null,
"_source": {
"syslog_severity_code": 5,
"offset": 41691,
"syslog_facility": "user-level",
"input_type": "log",
"syslog_facility_code": 1,
"source": "/var/log/messages",
"message": "Aug 17 23:37:42 <security.notice> Server auditd[637]: renamed /var/audit/20170817133611.not_terminated to /var/audit/20170817133611.20170817133742",
"type": "syslog",
"syslog_severity": "notice",
"tags": [
"beats_input_codec_plain_applied",
"_grokparsefailure"
],
"@timestamp": "2017-08-17T13:37:46.318Z",
"@version": "1",
"beat": {
"hostname": "Server.local",
"name": "Server.local",
"version": "5.5.1"
},
"host": "Server.local",
"fields": {
"system_type": "File Server",
"operating_system": "FreeBSD"
}
},
"fields": {
"@timestamp": [
1502977066318
]
},
"sort": [
1502977066318
]
}

I know why the filter doesn't work. FreeBSD syslog messages look like the following:

Aug 17 23:37:42 <security.notice> Server auditd[637]: renamed /var/audit/20170817133611.not_terminated to /var/audit/20170817133611.20170817133742

The facility and severity are not being correctly parsed. the are located between the <>. I'm not sure how to fix the grok filter to accomodate this.

Can anyone help me fix my filter to correctly parse FreeBSD syslogs

Many thanks

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.