Parsing FreeBSD syslog

Elastic Stack Logstash
1

Hi,

I am trying to parse FreeBSD syslog. The syslogs are sent to logstash via filebeat. I currently use this grok:

filter {
if [type] == "syslog" {

grok {
  match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
  add_field => [ "received_at", "%{@timestamp}" ]
  add_field => [ "received_from", "%{host}" ]
}

syslog_pri { }

date {
  match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
}

}
}

and the output of this is:

{
"_index": "filebeat-2017.08.17",
"_type": "syslog",
"_id": "AV3wahQdJ57xNwGFxihH",
"_version": 1,
"_score": null,
"_source": {
"syslog_severity_code": 5,
"offset": 41691,
"syslog_facility": "user-level",
"input_type": "log",
"syslog_facility_code": 1,
"source": "/var/log/messages",
"message": "Aug 17 23:37:42 <security.notice> Server auditd[637]: renamed /var/audit/20170817133611.not_terminated to /var/audit/20170817133611.20170817133742",
"type": "syslog",
"syslog_severity": "notice",
"tags": [
"beats_input_codec_plain_applied",
"_grokparsefailure"
],
"@timestamp": "2017-08-17T13:37:46.318Z",
"@version": "1",
"beat": {
"hostname": "Server.local",
"name": "Server.local",
"version": "5.5.1"
},
"host": "Server.local",
"fields": {
"system_type": "File Server",
"operating_system": "FreeBSD"
}
},
"fields": {
"@timestamp": [
1502977066318
]
},
"sort": [
1502977066318
]
}

I know why the filter doesn't work. FreeBSD syslog messages look like the following:

Aug 17 23:37:42 <security.notice> Server auditd[637]: renamed /var/audit/20170817133611.not_terminated to /var/audit/20170817133611.20170817133742

The facility and severity are not being correctly parsed. the are located between the <>. I'm not sure how to fix the grok filter to accomodate this.

Can anyone help me fix my filter to correctly parse FreeBSD syslogs

Many thanks

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.