How to use \n as a delimeter in dissect filter

Elastic Stack Logstash
1

Hi All,

dissect filter doesn't work when I use \n as a delimiter, it return error as
Dissector mapping, pattern not found {"field"=>"message", "pattern"=>"%{test}\n%{test1}"

It works on any other delimeter but \n doesn't works.

Can anyone please suggest me work around of it

filter {
  dissect {
     mapping {
           "message" => "%{test}\n%{test1}"
     }
 }
}

String to be matched is like

aaaa\nAAAA\nBBBB

2

Try using a liternal newline in the filter.

dissect {
    mapping {
        "message" => "%{test}
%{test1}"
   }
}
3

Hi @Badger
It didn't work, now throwing error like

Failed to execute action {:id=>:splunk, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Expected one of [ \t\r\n], "#", "=>" at line 59, column 27 (byte 2658) after filter {\n ... dissect {\n mapping ", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/compiler.rb:32:in `compile_imperative'", "org/logstash/execution/AbstractPipelineExt.java:183:in `initialize'", "org/logstash/execution/JavaBasePipelineExt.java:69:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/reload.rb:53:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:357:in `block in converge_state'"]}

code seems exactly the same as you suggested

dissect {
mapping {
"message" => "%{test}

%{test1}"
}
}
image

4

That should be mapping => {, not mapping {

Also, it looks like you have a blank line in the middle of the mapping, which you probably do not want to.

5

Sir, actually I want to use double backslashes \n\n as a delimeter.

I am going to apply this on windows event log ID: 4624 which is going to be parsed to extract main portions of event logs like

dissect {
mapping => {
"message" => "%{?winlog_header}\n\n%{winlog_subject}\n\n%{winlog_logon}\n\n%{winlog_impersonation}\n\n%{winlog_newlogon}\n\n%{winlog_process}\n\n%{winlog_network}\n\n%{winlog_authentication}"
}
}

these main fields like winlog_subject, winlog_process will again be dissected with delimeter \n but the major portion has to be dissected with delimeter \n\n. That's why I showed here \n\n in the code and screenshot.

Thank you for your reply.

6

Hi Sir,

Can I use this like as being shown in the screenshot.

image
image440×643 12.9 KB

         dissect {
            mapping => {
              "message" => "%{?winlog_header}

              %{winlog_subject}

              %{winlog_logon}

              %{winlog_impersonation}

              %{winlog_newlogon}

              %{winlog_process}

              %{winlog_network}

              %{winlog_authentication}"
              "winlog_subject" => "%{?subject_header}
              %{winlog_eventdata_SubjectUserSid}
              %{winlog_eventdata_SubjectUserName}
              %{winlog_eventdata_SubjectDomainName}
              %{winlog_eventdata_SubjectLogonId}"
              "winlog_logon" => "%{?winlog_logon_header}
              %{winlog_eventdata_LogonType}
              %{?winlog_eventdata_RestrictedAdminMode}
              %{?winlog_eventdata_VirtualAccount}
              %{?winlog_eventdata_ElevatedToken}"
              "winlog_newlogon" => "%{?newlogon_header}
              %{winlog_eventdata_TargetUserSid}
              %{winlog_eventdata_TargetUserName}
              %{winlog_eventdata_TargetDomainName}
              %{winlog_eventdata_TargetLogonId}
              %{winlog_eventdata_LinkedLogonID}
              %{?winlog_eventdata_NetworkAccountName}
              %{winlog_eventdata_NetworkAccountDomain}
              %{winlog_eventdata_TargetLogonGuid}"
              "winlog_process" => "%{?process_header}
              %{winlog_eventdata_ProcessId}
              %{winlog_eventdata_ProcessName}"
              "winlog_network" => "%{?network_header}
              %{?winlog_eventdata_WorkstationName}
              %{winlog_eventdata_IpAddress}
              %{winlog_eventdata_IpPort}"
            }
          }#end_dissect
7

That could work if each line starts with 14 spaces, otherwise you may want those left aligned.

8

Sir, both of them didn't work. Above screenshot ^ and this one

dissect {
mapping => {
"message" => "%{?winlog_header}

%{winlog_subject}

%{winlog_logon}

%{winlog_impersonation}

%{winlog_newlogon}

%{winlog_process}

%{winlog_network}

%{winlog_authentication}"
"winlog_subject" => "%{?subject_header}
%{winlog_eventdata_SubjectUserSid}
%{winlog_eventdata_SubjectUserName}
%{winlog_eventdata_SubjectDomainName}
%{winlog_eventdata_SubjectLogonId}"
"winlog_logon" => "%{?winlog_logon_header}
%{winlog_eventdata_LogonType}
%{?winlog_eventdata_RestrictedAdminMode}
%{?winlog_eventdata_VirtualAccount}
%{?winlog_eventdata_ElevatedToken}"
"winlog_newlogon" => "%{?newlogon_header}
%{winlog_eventdata_TargetUserSid}
%{winlog_eventdata_TargetUserName}
%{winlog_eventdata_TargetDomainName}
%{winlog_eventdata_TargetLogonId}
%{winlog_eventdata_LinkedLogonID}
%{?winlog_eventdata_NetworkAccountName}
%{winlog_eventdata_NetworkAccountDomain}
%{winlog_eventdata_TargetLogonGuid}"
"winlog_process" => "%{?process_header}
%{winlog_eventdata_ProcessId}
%{winlog_eventdata_ProcessName}"
"winlog_network" => "%{?network_header}
%{?winlog_eventdata_WorkstationName}
%{winlog_eventdata_IpAddress}
%{winlog_eventdata_IpPort}"
}
}#end_dissect

Thowing error ..
][WARN ][org.logstash.dissect.Dissector][splunk][19a778015acfcec74d96e1de54ccc439245a8b4f8149804f46cbac81c9d9b8d3] Dissector mapping, pattern not found

9

I suggest you start with

then add one line at a time to the mapping to consume additional parts of the message.

10

Thank you. It's working fine

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.