I'm using logstash 2.2.2 with the syslog output on a centos 7 system. Logstash is receiving data from other servers forwarded by rsyslog. Using the file output results in data such as:
<85>Jul 14 08:49:07 td032 sudo: root : TTY=unknown ; PWD=/ ; USER=xxx; COMMAND=/usr/java/default//bin/jps -q
Using the syslog output the data is different:
2016-07-14T06:49:07.510+00:00 td032 sudo root : TTY=unknown ; PWD=/ ; USER=xxx; COMMAND=/usr/java/default//bin/jps -q
I am in this case concerned about the timestamp. Although they are essentially the same, I would really like to maintain the timezone +0200 that I'm in for the syslog output, just like the file output does. Is this possible? If not, can we add such functionality? Why is the behaviour different from the file output?
It seems like there is a timestamp option but it is being deprecated. Would it solve my problem? Why is it being deprecated?
This is the console output of logstash for the event:
{
"message" => "<85>Jul 14 08:49:07 td032 sudo root : TTY=unknown ; PWD=/ ; USER=xxx; COMMAND=/usr/java/default//bin/jps -q",
"@version" => "1",
"@timestamp" => "2016-07-14T06:49:07.510Z",
"host" => "###.##.##.##",
"port" => 42176,
"type" => "generic-syslog",
"syslog_pri" => "85",
"timestamp" => "Jul 14 08:49:07",
"logsource" => "td032",
"program" => "sudo",
"pid" => "15493",
"syslog_message" => " root : TTY=unknown ; PWD=/ ; USER=xxx; COMMAND=/usr/java/default//bin/jps -q",
"received_at" => "2016-07-14T06:49:07.510Z",
"received_from" => "###.##.##.##",
"syslog_severity_code" => 5,
"syslog_facility_code" => 13,
"syslog_facility" => "security",
"syslog_severity" => "notice"
}
This is the logstash configuration that I'm using:
input {
udp {
type => 'generic-syslog'
port => 5142
buffer_size => 65536
workers => 10
}
tcp {
type => 'generic-syslog'
port => 5142
}
}
filter {
# Detect what type of message it is, then tag it accordingly.
# Manual grok because of not using syslog input
grok {
# This match is based on the logstash pattern SYSLOGLINE but modified to
# better match our needs.
match => { "message" => "<%{POSINT:syslog_pri}>(?:%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp8601}) (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource}+(?: %{SYSLOGPROG}:|)(\s)%{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
# Decode the facility and priority of the message.
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
# if the PID was not found/set, make it a - for the syslog output to work properly towards rsyslog
if !([pid] =~ /.+/) {
mutate { add_field => [ "pid", "-" ] }
}
}
output {
if [type] == 'generic-syslog' {
stdout { codec => rubydebug }
}
if [type] == 'generic-syslog' {
file {
path => "/export/logs/debugging/send-to-syslog.log"
codec => line { format => "%{message}" }
flush_interval => "0"
}
syslog {
appname => "%{program}"
procid => "%{pid}"
message => "%{syslog_message}"
facility => "%{syslog_facility}"
severity => "%{syslog_severity}"
sourcehost => "%{logsource}"
host => "localhost"
port => 514
workers => 10
rfc => "rfc5424"
}
}
}