Correct use of dissect

[Callahan]

I've coded my input for some FW logs. I initially sent those logs to a simple syslog server so I could code my config for it. The syslog input looks like this:

10.50.255.34 Jan 8 13:38:09 firewall.hostname.com user info 1,2019/01/08 13:38:09,002201003331,CONFIG,0,0,2019/01/08 13:38:09,10.56.10.31,,commit,_firstname.lastname,Web,Submitted,,2716,0x0,0,0,0,0,,firewall

My config for that looks like this. Note that on one specific line there is a caveat for syslog entries that Logstash receives that contain the line "commit" in them. FOr these, I need to do a different dissect.

input {
  udp {
	 type => "config-changes"
	 port => 1729
	 }
}

filter {
  if [type] == "config-changes" {
    if "commit" not in [message] {
      dissect {
        mapping => {
          "message" => "%{?drop-ip->} %{?drop-month->} %{?drop-date} %{?drop-time1} %{?drop-host->} %{?drop-facility->} %{?drop-priority->} %{?drop-num1},%{@timestamp},%{drop-num2},%{type_of_change_1},%{drop-num3},%{drop-num4},%{drop-time2},%{FW_IP},%{?drop-field1},%{type_of_action},%{user},%{access_type},%{change_result}, %{details},%{?drop-field2},%{?drop-field3},%{?drop-field4},%{?drop-field5},%{?drop-field6},%{?drop-field7},%{?drop-field8},%{hostname}" 
        }
      }
    }  
    if "commit" in [message] {
      dissect {
        mapping => {
         "message" => "%{?drop-ip->} %{?drop-month->} %{?drop-date} %{?drop-time1} %{?drop-host->} %{?drop-facility->} %{?drop-priority->} %{?drop-num1},%{@timestamp},%{drop-num2},%{type_change_1},%{drop-num3},%{drop-num4},%{drop-time2},%{IP},%{?drop-field1},%{type_change_2},%{user},%{access_type},%{change_result},{?drop-field9},{?drop-field10},{?drop-field11},{?drop-field12},{?drop-field13},{?drop-field14},{?drop-field15},{?drop-field16},%{hostname}"
        }
      }
    }
  }
}
#End Filter block

output {
  if [type] == "config-changes" {
    elasticsearch {
      index => "config-changes-%{+YYYY.MM.dd}"
      hosts => ["elk.mydomain.com:9200"]
    }
  }
}

I'm guessing that [message] might not be the right way to check the incoming syslog messages. If it isn't, can you tell me how I check each incoming syslog for the "commit" or "not commit" lines?

Thanks.

[Badger]

In your sample data the fields before the first comma are tab separated, so you do not use ->. And several of the latter fields are missing the leading %. I would not name the skip fields, but if it helps you so be it. This works for me for the commit line.

    dissect {
        mapping => { "message" => "%{}      %{}     %{}     %{}     %{}     %{},%{ts},%{},%{type_change_1},%{},%{},%{},%{IP},%{},%{type_change_2},%{user},%{access_type},%{change_result},%{},%{},%{},%{},%{},%{},%{},%{},%{hostname}" }
    }

Parsing into @timestamp works, but you would then need to parse that using a date{} filter. Personally I would parse into a different field and then remove that in the date filter, or shove it under [@metadata]

Testing for substrings using "in" works fine. I happen to prefer =~, but either will work.