My inputs are a couple windows server, a linux server, and a Fortinet router. I am using Beats for the windows and linux servers. The Fortinet is forwarding it's logs. I am assigning a different type for each input (type => "the_type"). I then want to reference that type so I can apply different filters and use different output parameters based on the type.
Taking the config as posted, the "fortilog" type's index is created as "%{[@metadata][beat]}-2016.05.02" instead of "fortilog-2016.05.02". If I comment out the fortilog filter section, the index for the "fortilog" type is created properly (fortilog-2016.05.02). It feels like the filter is somehow stripping off the type.
input {
udp {
port => 5045
type => 'fortilog'
}
tcp {
port => 5045
type => 'fortilog'
}
beats {
port => 5044
ssl => true
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder2.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder2.key"
}
}
filter {
if [type] == 'fortilog' {
grok {
match => ["message" , "<%{NUMBER:syslog_index}>date=%{YEAR:year}-%{MONTHNUM:month}-%{MONTHDAY:day} time=%{TIME:time} devname=%{NOTSPACE:hostname} devid=%{HOSTNAME:devid} logid=%{NUMBER:logid} type=%{WORD:type} subtype=%{NOTSPACE:subtype}"]
}
}
else if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
else {}
}
output {
if [type] == 'fortilog' {
elasticsearch {
manage_template => false
hosts => ['localhost:9200']
index => '%{[type]}-%{+YYYY.MM.dd}'
}
}
else {
elasticsearch {
hosts => ["localhost:9200"]
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
}