How to write a Multiple input and logstash filter for hostname based on message field

As a newbie to logstash i would like to understand as i have two types of logs one is Linux system logs and another i have CISCO switches logs , now i'm looking forward to create the diffrent input and filter's for both.

I have defined the type for linux logs as syslog and for CISCO switches as APIC and want to define the and for filter section. My CISCO log pattrens sample is as below where my SWITCH NAME is 7th Field in the messages , so wonder how to take that 7th field as a Hostname for swiches.

Aug 23 16:36:58 Aug 23 11:06:58.830 mydc-leaf-3-5 %LOG_-1-SYSTEM_MSG [E4210472][transition][info][sys] sent user message to syslog group:Syslog_Elastic_Server:final

Blow is my logstash-syslog.conf file which is working for syslog but needs while for CISCO logs ie type => APIC ..

    # cat  logstash-syslog.conf
    input {
      file {
        path => [ "/scratch/rsyslog/*/messages.log" ]
        type => "syslog"
      }
    
      file {
        path => [ "/scratch/rsyslog/Aug/messages.log" ]
        type => "APIC"
      }
    }
    
    filter {
      if [type] == "syslog" {
        grok {
          match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
          add_field => [ "received_at", "%{@timestamp}" ]
          add_field => [ "received_from", "%{host}" ]
        }
        syslog_pri { }
        date {
          match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
        }
    
      else if [type] == "APIC" {
       if "%LOG_-3-SYSTEM_MSG" in [message] {
         grok {
           match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
           add_field => [ "received_at", "%{@timestamp}" ]
           add_field => [ "received_from", "%{host}" ]
        }
        syslog_pri { }
        date {
          match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
      }
    }
   }
    output {
            elasticsearch {
                    hosts => "noida-elk:9200"
                    index => "%{[type]}-%{+YYYY.MM.dd}"
                    document_type => "messages"
      }
    }

Filter works correctly for below message and i get the Field syslog_hostname correctly, here in case i get can get the linuxdev.

Aug 24 10:34:02 linuxdev automount[1905]: key ".P4Config" not found in map source(s).

Filter do not work for below message..

Aug 24 10:26:22 Aug 24 04:56:22.444 my-apic-1 %LOG_-3-SYSTEM_MSG [F1546][soaking_clearing][packets-dropped][minor][dbgs/ac/sdvpcpath-207-208-to-109-110/fault-F1546] 2% of packets were dropped during the last collection interval

My logstash version is 6.3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.