Problem with an IF statement not working

Elastic Stack Logstash
1

Hello Everyone,

I hope someone is able to assist. I am running ELK stack 8.11.3 and am using Logstash to ingest Syslogs in a CEF format. I have everything working and it works nicely, however I want to add an IF statement to mutate and rename certain field names IF the Field "cef_eventname" contains a specific string. It just DOES NOT want to work, and I can't figure out what I have done wrong.

This is my code. It uses GROK to filter into Regex Groups the syslog and CEF headers and then the message itself. The GROK plugin works fine and all data is grouped as I need it. Its the IF statement which is not!

filter {
     grok {
           match => { "message" => "(?<id><14>1) %{TIMESTAMP_ISO8601:timestamp} %{SYSLOGHOST:sysloghost} (?<twes>TW_ES) (?<dashes>\- \- \-) (?<cef_version>CEF:\d)\|(?<cef_vendor>\w*)\|(?<cef_deviceproduct>\w*)\|(?<cef_deviceversion>\d{0,9}\.\d{0,9})\|(?<cef_eventclassid>\d{0,9})\|(?<cef_eventname>\w.*)\|(?<cef_severity>\d+)\|%{GREEDYDATA:LogMessage}"
           }
         }  
       if "[cef_eventname]" == "File Integrity Change" {
	    mutate {
           rename => [ "[cs1]" , "[Cheese Toast]" ]
           add_tag => 	[ "IF works"]
			}
		  } 
	  	  #Parse the message with field=value formats
      kv { 
           field_split => "\|"
       }
	  mutate {
           rename => [ "[host][ip]" , "host_ip" ]
#		   rename => [ "cs1" , "Egg" ]
#          rename => [ "cs1" , "Node Type" ]
		   add_tag => [ "IF not worked"]
           }
   
}

Also, this is what the data looks like:

2024-01-22 12 55 08
2024-01-22 12 55 08265×816 22.1 KB

All help is appreciated.

2

Remove the double quotes on fields:
if [cef_eventname] == "File Integrity Change" {

3

I've tried this and it didn't seem to make a difference.

2024-01-23 13 34 54

I was thinking, if it could be something in my grok filter, but that is working correctly, because it is detecting and labelling the field ( as per my picture ) and the values are correct also.

I'm really puzzled!

Any help is much appreciated and thanks :slight_smile:

4

Can you share the json of this document in Kibana to show how the cef_eventname looks like?

Share the part inside the _source in the json view in discover, not the part under fields.

5

Sure @leandrojmp ,

Please see below:

  "_source": {
    "timezoneLabel": "Time Zone",
    "SHA-1": "Before=bf2378f33fdec10c34c20cd0a6d9c15baff0f17d;After=7c21261cd9a5e5177d9bc03a0e7167172f07400f",
    "Size": "Before=1350696;After=1353400",
    "cs4Label": "Change Type",
    "type": "syslog",
    "cef_vendor": "Tripwire",
    "cef_deviceversion": "5.5",
    "dvc": "172.23.180.89",
    "hardCodedIP": "10.42.1.51\r",
    "cn1Label": "Tripwire Severity Number",
    "cs3Label": "Rule Type",
    "cef_eventclassid": "1",
    "duser": "NT AUTHORITY\\SYSTEM",
    "id": "<14>1",
    "sysloghost": "te-poc-hyperv",
    "fname": "C:\\Windows\\System32\\winresume.efi",
    "cn1": "100",
    "elementOIDLabel": "Element OID",
    "cef_severity": "2",
    "blVersionLabel": "Is baseline version",
    "dhost": "te-poc-hyperv",
    "cef_deviceproduct": "Enterprise",
    "cs1": "Windows Server",
    "sproc": "C:\\Windows\\System32\\poqexec.exe",
    "cs3": "Windows File System Rule",
    "rt": "Jan 23 2024 13:43:15",
    "cs2": "System Configuration Files",
    "cs5": "High",
    "cs4": "Modified",
    "timezone": "Greenwich Mean Time",
    "cs6": "-1y2p0ij32e8ch:-1y2p0ij1vig8u",
    "twes": "TW_ES",
    "cs1Label": "Node Type",
    "cs6Label": "Version OID",
    "content": "Not available",
    "dvchost": "te-dm-01.mshome.net",
    "@version": "1",
    "event": {
      "original": "<14>1 2024-01-23T13:43:15.211Z te-poc-hyperv TW_ES - - - CEF:0|Tripwire|Enterprise|5.5|1|File Integrity Change|2|dvchost=te-dm-01.mshome.net|cs1=Windows Server|cs1Label=Node Type|cs2=System Configuration Files|cs2Label=Rule|cs3=Windows File System Rule|cs3Label=Rule Type|fname=C:\\Windows\\System32\\winresume.efi|cs4=Modified|cs4Label=Change Type|cs5=High|cs5Label=Tripwire Severity Name|cs6=-1y2p0ij32e8ch:-1y2p0ij1vig8u|cs6Label=Version OID|cn1=100|cn1Label=Tripwire Severity Number|sproc=C:\\Windows\\System32\\poqexec.exe|licurl=https://te-poc-hyperv/console/lic.search.cmd?lic=true&managerId=nodeManager&pageId=nodeManager.elementFinderPage&searchCriteria=%7B%22search.element.nodeGroup.selectedObject%22%3A%22-1y2p0ij32e8bv%3A-1y2p0ij1zzm0h%22%2C%22search.element.name.op%22%3A1%2C%22search.element.name%22%3A%22C%3A%5C%5CWindows%5C%5CSystem32%5C%5Cwinresume.efi%22%2C%22selectedSearchType%22%3A%22element%22%2C%22search.element.ruleGroup.selectedObject%22%3A%22-1y2p0ij32e7ps%3A-1y2p0ij1zzluk%22%2C%22criteria.searchExecuted%22%3Atrue%7D|start=Jan 23 2024 12:12:36|duser=NT AUTHORITY\\SYSTEM|dvc=172.23.180.89|rt=Jan 23 2024 13:43:15|dhost=te-poc-hyperv|SHA-1=Before=bf2378f33fdec10c34c20cd0a6d9c15baff0f17d;After=7c21261cd9a5e5177d9bc03a0e7167172f07400f|MD5=Not available|Size=Before=1350696;After=1353400|content=Not available|contentLabel=Current Version Content|timezone=Greenwich Mean Time|timezoneLabel=Time Zone|elementOID=-1y2p0ij32e8cc:-1y2p0ij1zxvxs|elementOIDLabel=Element OID|blVersion=false|blVersionLabel=Is baseline version|hardCodedIP=10.42.1.51\r"
    },
    "timestamp": "2024-01-23T13:43:15.211Z",
    "cs5Label": "Tripwire Severity Name",
    "cef_eventname": "File Integrity Change",
    "licurl": "https://te-poc-hyperv/console/lic.search.cmd?lic=true&managerId=nodeManager&pageId=nodeManager.elementFinderPage&searchCriteria=%7B%22search.element.nodeGroup.selectedObject%22%3A%22-1y2p0ij32e8bv%3A-1y2p0ij1zzm0h%22%2C%22search.element.name.op%22%3A1%2C%22search.element.name%22%3A%22C%3A%5C%5CWindows%5C%5CSystem32%5C%5Cwinresume.efi%22%2C%22selectedSearchType%22%3A%22element%22%2C%22search.element.ruleGroup.selectedObject%22%3A%22-1y2p0ij32e7ps%3A-1y2p0ij1zzluk%22%2C%22criteria.searchExecuted%22%3Atrue%7D",
    "start": "Jan 23 2024 12:12:36",
    "message": "<14>1 2024-01-23T13:43:15.211Z te-poc-hyperv TW_ES - - - CEF:0|Tripwire|Enterprise|5.5|1|File Integrity Change|2|dvchost=te-dm-01.mshome.net|cs1=Windows Server|cs1Label=Node Type|cs2=System Configuration Files|cs2Label=Rule|cs3=Windows File System Rule|cs3Label=Rule Type|fname=C:\\Windows\\System32\\winresume.efi|cs4=Modified|cs4Label=Change Type|cs5=High|cs5Label=Tripwire Severity Name|cs6=-1y2p0ij32e8ch:-1y2p0ij1vig8u|cs6Label=Version OID|cn1=100|cn1Label=Tripwire Severity Number|sproc=C:\\Windows\\System32\\poqexec.exe|licurl=https://te-poc-hyperv/console/lic.search.cmd?lic=true&managerId=nodeManager&pageId=nodeManager.elementFinderPage&searchCriteria=%7B%22search.element.nodeGroup.selectedObject%22%3A%22-1y2p0ij32e8bv%3A-1y2p0ij1zzm0h%22%2C%22search.element.name.op%22%3A1%2C%22search.element.name%22%3A%22C%3A%5C%5CWindows%5C%5CSystem32%5C%5Cwinresume.efi%22%2C%22selectedSearchType%22%3A%22element%22%2C%22search.element.ruleGroup.selectedObject%22%3A%22-1y2p0ij32e7ps%3A-1y2p0ij1zzluk%22%2C%22criteria.searchExecuted%22%3Atrue%7D|start=Jan 23 2024 12:12:36|duser=NT AUTHORITY\\SYSTEM|dvc=172.23.180.89|rt=Jan 23 2024 13:43:15|dhost=te-poc-hyperv|SHA-1=Before=bf2378f33fdec10c34c20cd0a6d9c15baff0f17d;After=7c21261cd9a5e5177d9bc03a0e7167172f07400f|MD5=Not available|Size=Before=1350696;After=1353400|content=Not available|contentLabel=Current Version Content|timezone=Greenwich Mean Time|timezoneLabel=Time Zone|elementOID=-1y2p0ij32e8cc:-1y2p0ij1zxvxs|elementOIDLabel=Element OID|blVersion=false|blVersionLabel=Is baseline version|hardCodedIP=10.42.1.51\r",
    "cef_version": "CEF:0",
    "dashes": "- - -",
    "blVersion": "false",
    "@timestamp": "2024-01-23T13:43:14.998575300Z",
    "LogMessage": "dvchost=te-dm-01.mshome.net|cs1=Windows Server|cs1Label=Node Type|cs2=System Configuration Files|cs2Label=Rule|cs3=Windows File System Rule|cs3Label=Rule Type|fname=C:\\Windows\\System32\\winresume.efi|cs4=Modified|cs4Label=Change Type|cs5=High|cs5Label=Tripwire Severity Name|cs6=-1y2p0ij32e8ch:-1y2p0ij1vig8u|cs6Label=Version OID|cn1=100|cn1Label=Tripwire Severity Number|sproc=C:\\Windows\\System32\\poqexec.exe|licurl=https://te-poc-hyperv/console/lic.search.cmd?lic=true&managerId=nodeManager&pageId=nodeManager.elementFinderPage&searchCriteria=%7B%22search.element.nodeGroup.selectedObject%22%3A%22-1y2p0ij32e8bv%3A-1y2p0ij1zzm0h%22%2C%22search.element.name.op%22%3A1%2C%22search.element.name%22%3A%22C%3A%5C%5CWindows%5C%5CSystem32%5C%5Cwinresume.efi%22%2C%22selectedSearchType%22%3A%22element%22%2C%22search.element.ruleGroup.selectedObject%22%3A%22-1y2p0ij32e7ps%3A-1y2p0ij1zzluk%22%2C%22criteria.searchExecuted%22%3Atrue%7D|start=Jan 23 2024 12:12:36|duser=NT AUTHORITY\\SYSTEM|dvc=172.23.180.89|rt=Jan 23 2024 13:43:15|dhost=te-poc-hyperv|SHA-1=Before=bf2378f33fdec10c34c20cd0a6d9c15baff0f17d;After=7c21261cd9a5e5177d9bc03a0e7167172f07400f|MD5=Not available|Size=Before=1350696;After=1353400|content=Not available|contentLabel=Current Version Content|timezone=Greenwich Mean Time|timezoneLabel=Time Zone|elementOID=-1y2p0ij32e8cc:-1y2p0ij1zxvxs|elementOIDLabel=Element OID|blVersion=false|blVersionLabel=Is baseline version|hardCodedIP=10.42.1.51\r",
    "data_stream": {
      "namespace": "default",
      "type": "logs",
      "dataset": "generic"
    },
    "contentLabel": "Current Version Content",
    "elementOID": "-1y2p0ij32e8cc:-1y2p0ij1zxvxs",
    "cs2Label": "Rule",
    "MD5": "Not available"
  },
6

Just to emphasize what Rios said, this is testing whether the literal string "[cef_eventname]" is equal to the literal string "File Integrity Change", which is never, ever going to be true. To test the value of the field use

if [cef_eventname] == "File Integrity Change" {
7

Hey Badger,

Thank you, I tried the suggestion made by Rios by removing the quotes and that didn't seem to make a difference either. It still won't run the if statement.

Any other troubleshooting or testing I can look at?

Thank you.

8

Enable ruby debug to see is there something related to data. However IF it works.

input {
  generator {
       message => "<14>1 2024-01-23T13:43:15.211Z te-poc-hyperv TW_ES - - - CEF:0|Tripwire|Enterprise|5.5|1|File Integrity Change|2|dvchost=te-dm-01.mshome.net|cs1=Windows Server|cs1Label=Node Type|cs2=System Configuration Files|cs2Label=Rule|cs3=Windows File System Rule|cs3Label=Rule Type|fname=C:\\Windows\\System32\\winresume.efi|cs4=Modified|cs4Label=Change Type|cs5=High|cs5Label=Tripwire Severity Name|cs6=-1y2p0ij32e8ch:-1y2p0ij1vig8u|cs6Label=Version OID|cn1=100|cn1Label=Tripwire Severity Number|sproc=C:\\Windows\\System32\\poqexec.exe|licurl=https://te-poc-hyperv/console/lic.search.cmd?lic=true&managerId=nodeManager&pageId=nodeManager.elementFinderPage&searchCriteria=%7B%22search.element.nodeGroup.selectedObject%22%3A%22-1y2p0ij32e8bv%3A-1y2p0ij1zzm0h%22%2C%22search.element.name.op%22%3A1%2C%22search.element.name%22%3A%22C%3A%5C%5CWindows%5C%5CSystem32%5C%5Cwinresume.efi%22%2C%22selectedSearchType%22%3A%22element%22%2C%22search.element.ruleGroup.selectedObject%22%3A%22-1y2p0ij32e7ps%3A-1y2p0ij1zzluk%22%2C%22criteria.searchExecuted%22%3Atrue%7D|start=Jan 23 2024 12:12:36|duser=NT AUTHORITY\\SYSTEM|dvc=172.23.180.89|rt=Jan 23 2024 13:43:15|dhost=te-poc-hyperv|SHA-1=Before=bf2378f33fdec10c34c20cd0a6d9c15baff0f17d;After=7c21261cd9a5e5177d9bc03a0e7167172f07400f|MD5=Not available|Size=Before=1350696;After=1353400|content=Not available|contentLabel=Current Version Content|timezone=Greenwich Mean Time|timezoneLabel=Time Zone|elementOID=-1y2p0ij32e8cc:-1y2p0ij1zxvxs|elementOIDLabel=Element OID|blVersion=false|blVersionLabel=Is baseline version|hardCodedIP=10.42.1.51\r"
	   count => 1
  }
 
}

filter {
 grok {
           match => { "message" => "(?<id><14>1) %{TIMESTAMP_ISO8601:timestamp} %{SYSLOGHOST:sysloghost} (?<twes>TW_ES) (?<dashes>\- \- \-) (?<cef_version>CEF:\d)\|(?<cef_vendor>\w*)\|(?<cef_deviceproduct>\w*)\|(?<cef_deviceversion>\d{0,9}\.\d{0,9})\|(?<cef_eventclassid>\d{0,9})\|(?<cef_eventname>\w.*)\|(?<cef_severity>\d+)\|%{GREEDYDATA:LogMessage}"
           }
         }  
 if [cef_eventname] == "File Integrity Change" {
	    mutate { add_tag => [ "IF works"] }
	}
	
	   mutate{   remove_field => ["location", "host",  "message", "event", "log", "LogMessage"] }
}
output {
   
    stdout {codec => rubydebug{ metadata => false}} #         
}

Result:

{
         "cef_severity" => "2",
          "cef_version" => "CEF:0",
                   "id" => "<14>1",
           "cef_vendor" => "Tripwire",
    "cef_deviceversion" => "5.5",
    "cef_deviceproduct" => "Enterprise",
             "@version" => "1",
           "@timestamp" => 2024-01-23T19:18:22.115257500Z,
           "sysloghost" => "te-poc-hyperv",
                 "twes" => "TW_ES",
               "dashes" => "- - -",
            "timestamp" => "2024-01-23T13:43:15.211Z",
        "cef_eventname" => "File Integrity Change",
     "cef_eventclassid" => "1",
                 "tags" => [
        [0] "IF works"
    ]
}
9

Hey Rios,

So i have just tried your code. I agree you are right, the IF statement does work. I even had an IF statement in the output and that has worked fine.

But I have just noticed something from your code. You don't have the field split in your code.

kv { 
       field_split => "\|"
         }

When you add that back in, thats when the rename on the field "cs1" doesn't work. The tag still appears but the rename does not work.

Is that the same for you?

Thanks

10

Looking with more attention to your pipeline, your rename will never work because the cs1 field does not exist at the moment the rename is applied.

The cs1 field is only created after the kv filter, also, you should use a source in the kv filter to make it more clear on what field the kv is being applied.

When your message arrives at this filter, the cs1 field does not exist in the message, so it will not execute the rename.

This is where the cs1 filter will be created, you should also use source => LogMessage to make clear that this is the field with your kv data.

Your conditional to change the cs1 field name needs to happen after the kv filter.

Also, the format of your rename is not correct, it should be a hash.

Try this filter pipeline, it should work:

filter {
    grok {
        match => { 
            "message" => "(?<id><14>1) %{TIMESTAMP_ISO8601:timestamp} %{SYSLOGHOST:sysloghost} (?<twes>TW_ES) (?<dashes>\- \- \-) (?<cef_version>CEF:\d)\|(?<cef_vendor>\w*)\|(?<cef_deviceproduct>\w*)\|(?<cef_deviceversion>\d{0,9}\.\d{0,9})\|(?<cef_eventclassid>\d{0,9})\|(?<cef_eventname>\w.*)\|(?<cef_severity>\d+)\|%{GREEDYDATA:LogMessage}"
        }
    }  
    kv {
        source => "LogMessage" 
        field_split => "\|"
    }
    if [cef_eventname] == "File Integrity Change" {
        mutate {
            rename => {
                "cs1" => "Cheese Toast"
            }
        }
    }
	mutate {
        rename => {
            "[host][ip]" => "host_ip"
        }
    }
   
}
11

Thank you all @leandrojmp for your help. This has worked nicely for me.

I did try the kv filter before the IF statement but it did not work, and I think that was due to a combination of your suggestion and that made by @Rios about the double quotes " " provide literal meaning to the comparison for the fields.

Really appreciate it and I am new to the ELK stack but have had an opportunity to learn other things from yourselves, like pumping sample data directly from the logstash config. (I did not know I could do this!)

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.