As a newbie to logstash i would like to understand as i have two types of logs one is Linux system logs and another i have CISCO switches logs , now i'm looking forward to create the diffrent input
and filter's
for both.
I have defined the type
for linux logs as syslog
and for CISCO
switches as APIC
and want to define the and for filter
section. My CISCO log pattrens sample is as below where my SWITCH NAME
is 7th Field in the messages , so wonder how to take that 7th field as a Hostname for swiches.
Aug 23 16:36:58 Aug 23 11:06:58.830 mydc-leaf-3-5 %LOG_-1-SYSTEM_MSG [E4210472][transition][info][sys] sent user message to syslog group:Syslog_Elastic_Server:final
Blow is my logstash-syslog.conf file which is working for syslog but needs while for CISCO logs ie type => APIC
..
# cat logstash-syslog.conf
input {
file {
path => [ "/scratch/rsyslog/*/messages.log" ]
type => "syslog"
}
file {
path => [ "/scratch/rsyslog/Aug/messages.log" ]
type => "APIC"
}
}
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
else if [type] == "APIC" {
if "%LOG_-3-SYSTEM_MSG" in [message] {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
output {
elasticsearch {
hosts => "noida-elk:9200"
index => "%{[type]}-%{+YYYY.MM.dd}"
document_type => "messages"
}
}
Filter works correctly for below message and i get the Field syslog_hostname
correctly, here in case i get can get the linuxdev
.
Aug 24 10:34:02 linuxdev automount[1905]: key ".P4Config" not found in map source(s).
Filter do not work for below message..
Aug 24 10:26:22 Aug 24 04:56:22.444 my-apic-1 %LOG_-3-SYSTEM_MSG [F1546][soaking_clearing][packets-dropped][minor][dbgs/ac/sdvpcpath-207-208-to-109-110/fault-F1546] 2% of packets were dropped during the last collection interval
My logstash version is 6.3