Parsing Symantec Enterprise Protection (SEPM) Syslog messages in logstash

Elastic Stack Logstash
1

SEPM sends syslog messages in batches at a time interval that is user specified. According to our SEP manager, it is not able to send them one at a time. The message field separates each message with the \r (carriage return) character.

I would like to separate each line out into its own event, and then do some post processing on it to parse out fields and standardize the timestamp (SEPM does not provide a year).

I've seen discussion posts about reading it into a Ruby array, and then separating it out, but I am unable to figure out how to do it.

  1. is the Ruby method the best way? If not, what is suggested?
  2. If Ruby, can you show how the filter would look to split the message on \r?

Thank you kindly in advance.

2

Is this on Windows or UNIX?

If UNIX, then you can enter literal Ctrl/M characters using Ctrl/V Ctrl/M. Then use mutate+split to convert the message to an array, and a split filter to split the array into multiple events.

input { generator { count => 1 message => 'foo^Mbar^Mbaz' } }

filter {
    mutate { split => { "message" => "^M" } }
    split { field => "message" }
}
3

Thank you for your reply! I was out yesterday and unable to test this.

The SEPM Server is Windows 2016, Elastic is running on Ubuntu 18.04 LTS.

Here is a much abbreviated (and sanitized) log sample that shows just two lines separated by the '\r'

> <50>Apr 15 10:03:50 SEPM SymantecServer: H2HWEB01,SHA-256: 0000000000000000000000000000000000000000000000000000000000000000,MD-5: ,[SID: 31358] Attack: ThinkPHP getShell Remote Code Execution 2 attack blocked. Traffic has been blocked for this application: SYSTEM,Local: 192.168.1.7,Local: 000000000000,Remote: ,Remote: 129.28.4.4,Remote: 000000000000,Inbound,TCP,Intrusion ID: 0,Begin: 2019-02-24 15:57:47,End: 2019-02-24 15:57:47,Occurrences: 1,Application: SYSTEM,Location: Default,User: used,Domain: TEST,Local Port 80,Remote Port 60726,CIDS Signature ID: 31358,CIDS Signature string: Attack: ThinkPHP getShell Remote Code Execution 2,CIDS Signature SubID: 76184,Intrusion URL: 199.48.152.1/index.php?s=captcha,Intrusion Payload URL: \r<50>Apr 15 10:03:50 SEPM SymantecServer: H2HWEB01,SHA-256: 0000000000000000000000000000000000000000000000000000000000000000,MD-5: ,[SID: 31358] Attack: ThinkPHP getShell Remote Code Execution 2 attack blocked. Traffic has been blocked for this application: SYSTEM,Local: 192.168.1.7,Local: 000000000000,Remote: ,Remote: 129.28.4.4,Remote: 000000000000,Inbound,TCP,Intrusion ID: 0,Begin: 2019-02-24 15:57:42,End: 2019-02-24 15:57:42,Occurrences: 1,Application: SYSTEM,Location: Default,User: used,Domain: TEST,Local Port 80,Remote Port 60084,CIDS Signature ID: 31358,CIDS Signature string: Attack: ThinkPHP getShell Remote Code Execution 2,CIDS Signature SubID: 76184,Intrusion URL: 199.48.152.1/index.php?s=captcha,Intrusion Payload URL:

The '\r' is at character 741, about halfway in.

I tried using \r, \r, and ^M in the mutate->split, but the messages are not split out into separate events. I am not sure I understand your use of Ctrl/V & Ctrl/M in your reply & expample.

Thanks!

4

^M is how a ctrl+M character is printed in a terminal. If you enter "stty -a" I expect that you will see "lnext = ^V" amongst the output. In an editor, in insert mode, if you type Enter, normally you would start a new line. lnext tells the tty to take the next character literally. So, in your editor, in insert mode, if you ctrl+V followed by Enter, you will end up with ^M in the file, which is what you need.

Another option is to set config.support_escapes to true in your logstash.yml, then you can use \r in the string.

5

Thank you again for your rapid response!

I am using managed pipelines, so this pipeline is edited through Kibana using Chrome. In this case, I should be able to just type in the ^M directly?

6

I do not know how to enter ^M in a browser. You may be forced into config.support_escapes

7

Thank you! the config.support_escapes did the trick.

following the last split command here, will any new_command apply to each split out event, so I can parse them further? Thanks again!

filter {
    mutate { split => { "message" => "\r" } }
    split { field => "message" }
    new_command {}
}
8

Yes, additional filters after the split will apply to each of the events that the split generates.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.