Syslog message: Tag problem and bad facility and severity values in the output of logstash

Elastic Stack Logstash
1

Hi
I have two virtual machines with centos 7 64 bits installed on it.
One is used as a server for log messages.
Logstach has been installed on it.
nagioslogserver version is 2015r1.4.

Here is the configuration file.

input {
syslog {
type => 'syslog'
port => 5544
}
tcp {
type => 'eventlog'
port => 3515
codec => json {
charset => 'CP1252'
}
}
tcp {
type => 'import_raw'
tags => 'import_raw'
port => 2056
}
tcp {
type => 'import_json'
tags => 'import_json'
port => 2057
codec => json
}
}

filter {
if [program] == 'apache_access' {
grok {
match => [ 'message', '%{COMBINEDAPACHELOG}']
}
date {
match => [ 'timestamp', 'dd/MMM/yyyy:HH:mm:ss Z' ]
}
mutate {
replace => [ 'type', 'apache_access' ]
convert => [ 'bytes', 'integer' ]
convert => [ 'response', 'integer' ]
}
}

if [program] == 'apache_error' {
    grok {
        match => [ 'message', '\[(?<timestamp>%{DAY:day} %{MONTH:month} %{MONTHDAY} %{TIME} %{YEAR})\] \[%{WORD:class}\] \[%{WORD:originator} %{IP:clientip}\] %{GREEDYDATA:errmsg}']
    }
    mutate {
        replace => [ 'type', 'apache_error' ]
    }
}

}
output {
elasticsearch {
cluster => 'c3adf935-f772-44c7-b39c-4fd7716317c0'
host => 'localhost'
index_type => '%{type}'
node_name => ''
protocol => 'transport'
workers => 4
}
if [facility] == 4 or [facility] == 10 or [facility] == 13 or [facility] == 14
{
file
{
path => "/var/log/logstash/audit_messages.log"
}
}
else
{
file
{
path => "/var/log/logstash/secure_messages.log"
}
}
}

From the other centos virtual machine, I send a rsyslog message with a facility equal to "auth" in order to test if the message is logged in the file /var/log/logstash/secure_messages.log as indicate in the logstash configuration file. In this machine, all rsyslog messages are forwarded to the logstash server.
rsyslog version is 7.4.7-6
I use a template in order to force RFC3164
Here is the configuration file for rsyslog:

### begin forwarding rule

NAGIOS LOG SERVER

$template RFC3164fmt,"<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag% %msg%"

EventTime
$WorkDirectory /var/lib/rsyslog # where to place spool files
$ActionQueueFileName fwdRule1 # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList # run asynchronously
$ActionResumeRetryCount -1 # infinite retries if host is down
. @localhost:5544;RFC3164fmt

### end of the forwarding rule

So I used the following command in order to generate rsysog messages:
logger -p auth.crit -t "logger" "**this is the test"

I expect to have my message written in the secure_messages.log file. However it was written in the audit_messages.log like this:

{"message":"<34>Nov 3 14:44:26 v2_SupervisionSSI logger: ***this is the test","@version":"1","@timestamp":"2015-11-03T13:44:26.896Z","type":"syslog","host":"192.168.8.25","tags":["_grokparsefailure"],"priority":13,"severity":5,"facility":1,"facility_label":"user-level","severity_label":"Notice","@fields":{}}

It seems there is a problem with the tag. Moreover the facility and severity are wrong( respectively 1 and 5 instead of 4 et 2).

So in the dashboard, the informations are wrong too.

In wireshark, the syslog messages seems good and matches with the template of rsyslog in the configuration file.

I note that any syslog message from any process generates the same errors on logstash.

What is wrong in my configuration?
Is the tag wrong?

Any help would be appreciated.

Thanks in advanced.

JF

2

I had similar frustrations with the syslog input. I just use tcp or udp input with my own grok. Can see here for a start but update the grok pattern accordingly.