How to add a pattern variable for all messages

[gcranshaw (Gregg Cranshaw)]

OK, I'm at a loss on how I can accomplish this. I am using the following logstash 5.2 filter for Cisco ASA's.

filter {
  if [type] == "cisco-asa" {
    grok {
      patterns_dir => ["/opt/logstash/patterns"]
      match => [
        "message", "%{CISCOFW106001}",
        "message", "%{CISCOFW106006_106007_106010}",
        "message", "%{CISCOFW106014}",
        "message", "%{CISCOFW106015}",
        "message", "%{CISCOFW106021}",
        "message", "%{CISCOFW106023}",
        "message", "%{CISCOFW106100}",
        "message", "%{CISCOFW110002}",
        "message", "%{CISCOFW302010}",
        "message", "%{CISCOFW302013_302014_302015_302016}",
        "message", "%{CISCOFW302020_302021}",
        "message", "%{CISCOFW305011}",
        "message", "%{CISCOFW313001_313004_313008}",
        "message", "%{CISCOFW313005}",
        "message", "%{CISCOFW402117}",
        "message", "%{CISCOFW402119}",
        "message", "%{CISCOFW419001}",
        "message", "%{CISCOFW419002}",
        "message", "%{CISCOFW500004}",
        "message", "%{CISCOFW602303_602304}",
        "message", "%{CISCOFW710001_710002_710003_710005_710006}",
        "message", "%{CISCOFW713172}",
        "message", "%{CISCOFW733100}",
        # Additional patterns not included in the core
        "message", "%{CISCOFW713050}",
        "message", "%{CISCOFW713259}",
        "message", "%{CISCOFW113019}",
        "message", "%{CISCOFW713119}",
        "message", "%{CISCOFW713120}",
        "message", "%{CISCOFW713049}",
        "message", "%{CISCOFW713076}",
        "message", "%{CISCOFW713130}"
      ]
    }
    syslog_pri {
    }
    geoip {
      source => "src_ip"
      target => "geoip"
      database => "/data/geoipdb/GeoLite2-City.mmdb"
    }
  }
}

This filter uses the core patterns found in Logstash and also points to the below additional patterns file.

#== Additional Cisco ASA patterns ==#

# Common Particles

# ASA-5-713050
CISCOFW713050 Group = %{GREEDYDATA:vpngroup}, Username = %{GREEDYDATA:vpnuser}, IP = %{IP:src_ip}, %{GREEDYDATA:vpnmessage}  Reason: %{GREEDYDATA:vpnreason}  Remote Proxy %{GREEDYDATA:vpnremoteprox}, Local Proxy %{GREEDYDATA:vpnlocalprox}

# ASA-5-713259
CISCOFW713259 Group = %{GREEDYDATA:vpngroup}, Username = %{GREEDYDATA:vpnuser}, IP = %{IP:src_ip}, %{GREEDYDATA:vpnmessage} Reason: %{GREEDYDATA:vpnreason}

# ASA-4-113019
CISCOFW113019 Group = %{GREEDYDATA:vpngroup}, Username = %{GREEDYDATA:vpnuser}, IP = %{IP:src_ip}, %{GREEDYDATA:vpnmessage} Session Type: %{GREEDYDATA:vpnsessiontype}, Duration: %{HOUR:vpndurationhour}h:%{MINUTE:vpndurationmin}m:%{SECOND:vpndurationsec}s, Bytes xmt: %{GREEDYDATA:vpnxmtbytes:int}, Bytes rcv: %{GREEDYDATA:vpnrcvbytes:int}, Reason: %{GREEDYDATA:vpnreason}

# ASA-5-713119
CISCOFW713119 Group = %{GREEDYDATA:vpngroup}, Username = %{GREEDYDATA:vpnuser}, IP = %{IP:src_ip}, %{GREEDYDATA:vpnmessage}

# ASA-5-713120
CISCOFW713120 Group = %{GREEDYDATA:vpngroup}, Username = %{GREEDYDATA:vpnuser}, IP = %{IP:src_ip}, %{GREEDYDATA:vpnmessage}

# ASA-5-713049
CISCOFW713049 Group = %{GREEDYDATA:vpngroup}, Username = %{GREEDYDATA:vpnuser}, IP = %{IP:src_ip}, %{GREEDYDATA:vpnmessage}  Responder, Inbound SPI = 0x7df633f6, Outbound SPI = 0x085c6667

# ASA-5-713076
CISCOFW713076 Group = %{GREEDYDATA:vpngroup}, Username = %{GREEDYDATA:vpnuser}, IP = %{IP:src_ip}, %{GREEDYDATA:vpnmessage}

# ASA-5-713130
CISCOFW713130 Group = %{GREEDYDATA:vpngroup}, Username = %{GREEDYDATA:vpnuser}, IP = %{IP:src_ip}, %{GREEDYDATA:vpnmessage}

What I want to do is pull a piece of data found in every log message and have it indexed. Below is an example log file with some data removed from it.

Feb 16 15:29:07 x.x.x.x %ASA-2-106001: Inbound TCP connection denied from x.x.x.x/1234 to x.x.x.x/1234 flags SYN on interface outside

I want the below data outlined with the grok variable.

Feb 16 15:29:07 %{IP:ciscoasahost} %ASA-2-106001: Inbound TCP connection denied from x.x.x.x/1234 to x.x.x.x/1234 flags SYN on interface outside

I know using the patterns I created above I could add an entry in each log type for this, but that would only cover each unique log I defined in my patterns file. Each unique log defined in the core patterns file could not be modified. Also, I'm not even sure how I could add a grok variable for data that comes before the pattern definition name, which is the ASA-#-###### number.

I'm sure this is an easy fix, but so far I have not found anything that works and I am a Logstash/grok newbie. Any help would be greatly appreciated.

[gcranshaw (Gregg Cranshaw)]

I figured it out. I had to add an additional grok entry into the configuration file.

    grok {
  patterns_dir => ["/opt/logstash/patterns"]
  match => [
    "message", "%{CISCOLOGDETAILS}"
  ]
}

And add the below pattern into the patterns file.

CISCOLOGDETAILS %{IP:ciscohost} %%{GREEDYDATA:ciscotagnum}:

This seemed to allow adding the additional items to the already added data from the patterns already defined.