How to use \n as a delimeter in dissect filter

Elastic Stack Logstash
1

Hi All,

dissect filter doesn't work when I use \n as a delimiter, it return error as
Dissector mapping, pattern not found {"field"=>"message", "pattern"=>"%{test}\n%{test1}"

It works on any other delimeter but \n doesn't works.

Can anyone please suggest me work around of it

filter {
  dissect {
     mapping {
           "message" => "%{test}\n%{test1}"
     }
 }
}

String to be matched is like

aaaa\nAAAA\nBBBB

2

Try using a liternal newline in the filter.

dissect {
    mapping {
        "message" => "%{test}
%{test1}"
   }
}
3

Hi @Badger
It didn't work, now throwing error like

Failed to execute action {:id=>:splunk, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Expected one of [ \t\r\n], "#", "=>" at line 59, column 27 (byte 2658) after filter {\n ... dissect {\n mapping ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:183:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:69:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/reload.rb:53:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:357:in `block in converge_state'"]}

code seems exactly the same as you suggested

dissect {
mapping {
"message" => "%{test}

%{test1}"
}
}
image

4

That should be mapping => {, not mapping {

Also, it looks like you have a blank line in the middle of the mapping, which you probably do not want to.

5

Sir, actually I want to use double backslashes \n\n as a delimeter.

I am going to apply this on windows event log ID: 4624 which is going to be parsed to extract main portions of event logs like

dissect {
mapping => {
"message" => "%{?winlog_header}\n\n%{winlog_subject}\n\n%{winlog_logon}\n\n%{winlog_impersonation}\n\n%{winlog_newlogon}\n\n%{winlog_process}\n\n%{winlog_network}\n\n%{winlog_authentication}"
}
}

these main fields like winlog_subject, winlog_process will again be dissected with delimeter \n but the major portion has to be dissected with delimeter \n\n. That's why I showed here \n\n in the code and screenshot.

Thank you for your reply.

6

Hi Sir,

Can I use this like as being shown in the screenshot.

image
image440×643 12.9 KB

         dissect {
            mapping => {
              "message" => "%{?winlog_header}

              %{winlog_subject}

              %{winlog_logon}

              %{winlog_impersonation}

              %{winlog_newlogon}

              %{winlog_process}

              %{winlog_network}

              %{winlog_authentication}"
              "winlog_subject" => "%{?subject_header}
              %{winlog_eventdata_SubjectUserSid}
              %{winlog_eventdata_SubjectUserName}
              %{winlog_eventdata_SubjectDomainName}
              %{winlog_eventdata_SubjectLogonId}"
              "winlog_logon" => "%{?winlog_logon_header}
              %{winlog_eventdata_LogonType}
              %{?winlog_eventdata_RestrictedAdminMode}
              %{?winlog_eventdata_VirtualAccount}
              %{?winlog_eventdata_ElevatedToken}"
              "winlog_newlogon" => "%{?newlogon_header}
              %{winlog_eventdata_TargetUserSid}
              %{winlog_eventdata_TargetUserName}
              %{winlog_eventdata_TargetDomainName}
              %{winlog_eventdata_TargetLogonId}
              %{winlog_eventdata_LinkedLogonID}
              %{?winlog_eventdata_NetworkAccountName}
              %{winlog_eventdata_NetworkAccountDomain}
              %{winlog_eventdata_TargetLogonGuid}"
              "winlog_process" => "%{?process_header}
              %{winlog_eventdata_ProcessId}
              %{winlog_eventdata_ProcessName}"
              "winlog_network" => "%{?network_header}
              %{?winlog_eventdata_WorkstationName}
              %{winlog_eventdata_IpAddress}
              %{winlog_eventdata_IpPort}"
            }
          }#end_dissect
7

That could work if each line starts with 14 spaces, otherwise you may want those left aligned.

8

Sir, both of them didn't work. Above screenshot ^ and this one

dissect {
mapping => {
"message" => "%{?winlog_header}

%{winlog_subject}

%{winlog_logon}

%{winlog_impersonation}

%{winlog_newlogon}

%{winlog_process}

%{winlog_network}

%{winlog_authentication}"
"winlog_subject" => "%{?subject_header}
%{winlog_eventdata_SubjectUserSid}
%{winlog_eventdata_SubjectUserName}
%{winlog_eventdata_SubjectDomainName}
%{winlog_eventdata_SubjectLogonId}"
"winlog_logon" => "%{?winlog_logon_header}
%{winlog_eventdata_LogonType}
%{?winlog_eventdata_RestrictedAdminMode}
%{?winlog_eventdata_VirtualAccount}
%{?winlog_eventdata_ElevatedToken}"
"winlog_newlogon" => "%{?newlogon_header}
%{winlog_eventdata_TargetUserSid}
%{winlog_eventdata_TargetUserName}
%{winlog_eventdata_TargetDomainName}
%{winlog_eventdata_TargetLogonId}
%{winlog_eventdata_LinkedLogonID}
%{?winlog_eventdata_NetworkAccountName}
%{winlog_eventdata_NetworkAccountDomain}
%{winlog_eventdata_TargetLogonGuid}"
"winlog_process" => "%{?process_header}
%{winlog_eventdata_ProcessId}
%{winlog_eventdata_ProcessName}"
"winlog_network" => "%{?network_header}
%{?winlog_eventdata_WorkstationName}
%{winlog_eventdata_IpAddress}
%{winlog_eventdata_IpPort}"
}
}#end_dissect

Thowing error ..
][WARN ][org.logstash.dissect.Dissector][splunk][19a778015acfcec74d96e1de54ccc439245a8b4f8149804f46cbac81c9d9b8d3] Dissector mapping, pattern not found

9

I suggest you start with

then add one line at a time to the mapping to consume additional parts of the message.

10

Thank you. It's working fine