Parsing problem with both csv and dissect

Elastic Stack Logstash
1

for now we have snare sending windows events to a rsyslog server which then sends in json to my elastic stack. The delimiter is #011.

---example log ---

Jul 23 03:24:19 dalaptop.mydomain.local MSWinEventLog#0111#011Security#01155513#011Mon Jul 23 03:24:19 2019#0114673#011Microsoft-Windows-Security-Auditing#011MYNET\DALAPTOP$#011N/A#011Success Audit#011dalaptop.mydomain.local#011Sensitive Privilege Use#011#011A privileged service was called....<body of text without #011 in it>..Privileges: SeTCbPrivilege#01155512

Should parse like this with CSV or dissect

Jul 23 03:24:19 dalaptop.mydomain.local MSWinEventLog
1
Security
55513
Mon Jul 23 03:24:19 2019
4673
Microsoft-Windows-Security-Auditing
MYNET\DALAPTOP$
N/A
Success Audit
dalaptop.mydomain.local
Sensitive Privilege Use
--
A privileged service was called....<body of text without #011 in it>..Privileges: SeTCbPrivilege
55512

Actually splits like this.

Jul 23 03:24:19 2019
4673
Microsoft-Windows-Security-Auditing
MYNET\DALAPTOP$
N/A
Success Audit
dalaptop.mydomain.local
Sensitive Privilege Use
--
A privileged service was called....<body of text without #011 in it>..Privileges: SeTCbPrivilege
55512

CSV usage.
columns => ["col1","col2","col3","col4"..."col15"]
separator => "#011"

dissect usage.
mapping => {
"message" => "%{col1}#011%{col2}#011{col3}#011{col4}...#011%{col15}" }

both CSV and dissect mis-divides this entirely and doesn't even show it.
Jul 23 03:24:19 dalaptop.mydomain.local MSWinEventLog#0111#011Security#01155513#011Mon
every field after that dissects or csv parses just fine.

Any idea why this is happening?

2

You are missing the % on some of those {} patterns. A properly formed filter works just fine provided the text that should not contain #011 does not contain #011.

      "col2" => "1",
      "col4" => "55513",
     "col12" => "Sensitive Privilege Use",
      "col3" => "Security",
     "col11" => "dalaptop.mydomain.local",
     "col13" => "",
      "col1" => "Jul 23 03:24:19 dalaptop.mydomain.local MSWinEventLog",
      "col9" => "N/A",
      "col5" => "Mon Jul 23 03:24:19 2019",
     "col10" => "Success Audit",
     "col14" => "A privileged service was called....<body of text without 011 in it>..Privileges: SeTCbPrivilege",
      "col6" => "4673",
      "col8" => "MYNET\\DALAPTOP$",
     "col15" => "55512",
      "col7" => "Microsoft-Windows-Security-Auditing"
3

Those missing % were me just now, my network is air-gapped so I manually typed in all of it. I had originally missed a couple when doing the dissect filter but found and fixed those. CSV and Dissect both skip over the first four separators.

I wanted to add that I don't have any drops in my filter anywhere.

4

finally figured out what was going on. it wasn't with CSV or Dissect. it was in rsyslog's configuration. I haven't completely figured it out but I have a workaround on it.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.