How to add a pattern variable for all messages

OK, I'm at a loss on how I can accomplish this. I am using the following logstash 5.2 filter for Cisco ASA's.

filter {
  if [type] == "cisco-asa" {
    grok {
      patterns_dir => ["/opt/logstash/patterns"]
      match => [
        "message", "%{CISCOFW106001}",
        "message", "%{CISCOFW106006_106007_106010}",
        "message", "%{CISCOFW106014}",
        "message", "%{CISCOFW106015}",
        "message", "%{CISCOFW106021}",
        "message", "%{CISCOFW106023}",
        "message", "%{CISCOFW106100}",
        "message", "%{CISCOFW110002}",
        "message", "%{CISCOFW302010}",
        "message", "%{CISCOFW302013_302014_302015_302016}",
        "message", "%{CISCOFW302020_302021}",
        "message", "%{CISCOFW305011}",
        "message", "%{CISCOFW313001_313004_313008}",
        "message", "%{CISCOFW313005}",
        "message", "%{CISCOFW402117}",
        "message", "%{CISCOFW402119}",
        "message", "%{CISCOFW419001}",
        "message", "%{CISCOFW419002}",
        "message", "%{CISCOFW500004}",
        "message", "%{CISCOFW602303_602304}",
        "message", "%{CISCOFW710001_710002_710003_710005_710006}",
        "message", "%{CISCOFW713172}",
        "message", "%{CISCOFW733100}",
        # Additional patterns not included in the core
        "message", "%{CISCOFW713050}",
        "message", "%{CISCOFW713259}",
        "message", "%{CISCOFW113019}",
        "message", "%{CISCOFW713119}",
        "message", "%{CISCOFW713120}",
        "message", "%{CISCOFW713049}",
        "message", "%{CISCOFW713076}",
        "message", "%{CISCOFW713130}"
    syslog_pri {
    geoip {
      source => "src_ip"
      target => "geoip"
      database => "/data/geoipdb/GeoLite2-City.mmdb"

This filter uses the core patterns found in Logstash and also points to the below additional patterns file.

#== Additional Cisco ASA patterns ==#

# Common Particles

# ASA-5-713050
CISCOFW713050 Group = %{GREEDYDATA:vpngroup}, Username = %{GREEDYDATA:vpnuser}, IP = %{IP:src_ip}, %{GREEDYDATA:vpnmessage}  Reason: %{GREEDYDATA:vpnreason}  Remote Proxy %{GREEDYDATA:vpnremoteprox}, Local Proxy %{GREEDYDATA:vpnlocalprox}

# ASA-5-713259
CISCOFW713259 Group = %{GREEDYDATA:vpngroup}, Username = %{GREEDYDATA:vpnuser}, IP = %{IP:src_ip}, %{GREEDYDATA:vpnmessage} Reason: %{GREEDYDATA:vpnreason}

# ASA-4-113019
CISCOFW113019 Group = %{GREEDYDATA:vpngroup}, Username = %{GREEDYDATA:vpnuser}, IP = %{IP:src_ip}, %{GREEDYDATA:vpnmessage} Session Type: %{GREEDYDATA:vpnsessiontype}, Duration: %{HOUR:vpndurationhour}h:%{MINUTE:vpndurationmin}m:%{SECOND:vpndurationsec}s, Bytes xmt: %{GREEDYDATA:vpnxmtbytes:int}, Bytes rcv: %{GREEDYDATA:vpnrcvbytes:int}, Reason: %{GREEDYDATA:vpnreason}

# ASA-5-713119
CISCOFW713119 Group = %{GREEDYDATA:vpngroup}, Username = %{GREEDYDATA:vpnuser}, IP = %{IP:src_ip}, %{GREEDYDATA:vpnmessage}

# ASA-5-713120
CISCOFW713120 Group = %{GREEDYDATA:vpngroup}, Username = %{GREEDYDATA:vpnuser}, IP = %{IP:src_ip}, %{GREEDYDATA:vpnmessage}

# ASA-5-713049
CISCOFW713049 Group = %{GREEDYDATA:vpngroup}, Username = %{GREEDYDATA:vpnuser}, IP = %{IP:src_ip}, %{GREEDYDATA:vpnmessage}  Responder, Inbound SPI = 0x7df633f6, Outbound SPI = 0x085c6667

# ASA-5-713076
CISCOFW713076 Group = %{GREEDYDATA:vpngroup}, Username = %{GREEDYDATA:vpnuser}, IP = %{IP:src_ip}, %{GREEDYDATA:vpnmessage}

# ASA-5-713130
CISCOFW713130 Group = %{GREEDYDATA:vpngroup}, Username = %{GREEDYDATA:vpnuser}, IP = %{IP:src_ip}, %{GREEDYDATA:vpnmessage}

What I want to do is pull a piece of data found in every log message and have it indexed. Below is an example log file with some data removed from it.

Feb 16 15:29:07 x.x.x.x %ASA-2-106001: Inbound TCP connection denied from x.x.x.x/1234 to x.x.x.x/1234 flags SYN on interface outside

I want the below data outlined with the grok variable.

Feb 16 15:29:07 %{IP:ciscoasahost} %ASA-2-106001: Inbound TCP connection denied from x.x.x.x/1234 to x.x.x.x/1234 flags SYN on interface outside

I know using the patterns I created above I could add an entry in each log type for this, but that would only cover each unique log I defined in my patterns file. Each unique log defined in the core patterns file could not be modified. Also, I'm not even sure how I could add a grok variable for data that comes before the pattern definition name, which is the ASA-#-###### number.

I'm sure this is an easy fix, but so far I have not found anything that works and I am a Logstash/grok newbie. Any help would be greatly appreciated.

I figured it out. I had to add an additional grok entry into the configuration file.

    grok {
  patterns_dir => ["/opt/logstash/patterns"]
  match => [
    "message", "%{CISCOLOGDETAILS}"

And add the below pattern into the patterns file.

CISCOLOGDETAILS %{IP:ciscohost} %%{GREEDYDATA:ciscotagnum}:

This seemed to allow adding the additional items to the already added data from the patterns already defined.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.