Logstash ternary condition when a winlogbeat field doesn't exist

[mguttula (Mohan G)]

Logstash is sending winlogbeat fields to Spunk and when the fields don't exist it is send something like %{[winlog][event_data][AuthenticationPackageName]}

I am trying to find a way to either not send the field if it doesn't exist or at least send something like " - " for the fields that don't exist.

I tried the following scenario:

scenario 1:

filter {
  if [agent][type] == "winlogbeat" {
    mutate { 
      add_field => { 
        "Authentication_package" => "%{[winlog][event_data][AuthenticationPackageName]}"
        "Error_Code" => "%{[winlog][event_data][ErrorCode]}"
      }
	  add_tag => [ "winsplunk" ]
    }
    ruby {

      init => "SplunkFields = ['[Authentication_package]','[Elevated_Token]']"
	  code => "
        SplunkFields.each { |k, v| 
		  if v.to_s.start_with?('%') || v =~ '%{['
		    event.set(k, 'hello')
          end
        }
	  "
    }
  }
}

Not sure how to get this work as there 15 or more fields sending to Splunk and 13 of the have "%{[" based on how many fields exists

scenario 2:

filter {
  if [agent][type] == "winlogbeat" {
    if ([winlog][event_data][AuthenticationPackageName]) {
      mutate { 
        add_field => { 
          "Authentication_package" => "%{[winlog][event_data][AuthenticationPackageName]}" 
        } 
      }
	} else {
      mutate { 
        add_field => { 
          "Authentication_package" => " - " 
        } 
      }      
    }
}

The problem with this scenario is I have to repeat this approach for 15 fields

[Badger]

If they are all top level fields then you can do this using a prune filter. Set blacklist_values to the default regexp for blacklist_names.

[mguttula (Mohan G)]

snippet of code in output section for Splunk where I could not check if fields have "%{["

      http {
	    url => "https://localhost:8088/services/collector"
        format => "json"
        http_method => "post"      
        headers => ["Authorization", "Splunk 3axxxx-xxxxxxx"]
        cacert => "/etc/logstash/certs/splunkcacert.pem"
        mapping => {
		  "event" => {
            "index" => "windows_os"
            "time"  => "%{[event][created]}"
            "dvc"   => "%{[host][name]}"
            "Authentication_package" => "%{Authentication_package}"
	    "Error_Code" => "%{Error_Code}"
          }
        }
      }

[mguttula (Mohan G)]

Below are some of the fields I adding in mutate plugin in filter section and sending these fields to Splunk in output section:

        "Authentication_package" => "%{[winlog][event_data][AuthenticationPackageName]}"
        "Elevated_Token" => "%{[winlog][event_data][ElevatedToken]}"
        "Error_Code" => "%{[winlog][event_data][ErrorCode]}"
        "Impersonation_Level" => "%{[winlog][event_data][ImpersonationLevel]}"
        "Key_Length" => "%{[winlog][event_data][KeyLength]}"
        "Linked_Logon_ID" => "%{[winlog][event_data][TargetLinkedLogonId]}"
        "Logon_Process" => "%{[winlog][event_data][LogonProcessName]}"
        "Logon_Type" => "%{[winlog][event_data][LogonType]}"
        "Package_Name_NTLM_only" => "%{[winlog][event_data][LmPackageName]}"
        "Restricted_Admin_Mode" => "%{[winlog][event_data][RestrictedAdminMode]}"
        "Security_ID" => "%{[winlog][event_data][param26]}"
        "Source_Network_Address" => "%{[winlog][event_data][IpAddress]}"
        "Source_Port" => "%{[winlog][event_data][IpPort]}"
        "Transmitted_Services" => "%{[winlog][event_data][TransmittedServices]}"
        "Creator_process_ID" => "%{[winlog][event_data][ProcessId]}"
        "Process_Command_Line" => "%{[process][command_line]}"			
        "Token_Elevation_Type" => "%{[winlog][event_data][TokenElevationType]}"

[mguttula (Mohan G)]

below code snippet is replacing fields starting with '%{[' to ' - '

	  code => "
        SplunkFields.each { |k, v| 
		  if event.get(k).start_with?('%{[')
		    event.set(k, ' - ')
          end
        }
	  "

But is this right way to send fields to Splunk? I wish I don't have to initialize in filter section if possible and don't even send the field to Splunk if it doesn't exist for a given message.