How to keep syslog header when saving to file?

[dickens88 (Charles Liu)]

Guys, when I use logstash to input syslog and keep the message to file, I found it seems logstash won't keep syslog header.

my configuration looks like this:

input{
     syslog {
        port => 1522
     }
}
output{
    file {
        path => "/log/proxy/%{+yyyyMMdd}/%{host}/%{+HH}.log"
        codec => line { format => "%{message}" }
    }
}

and the raw data I captured via tcpdump looks like this:

..........      <30>Aug  7 08:47:12 blrmwg01 mwg: CEF:0|McAfee|Web Gateway|7.7.2.5.0|200|Proxy-Enable Web Cache|2|rt=Aug 07 2019 08:47:12 cat=Access Log dst=x ...

but when I checked the message from file, it looks like this:

CEF:0|McAfee|Web Gateway|7.7.2.5.0|200|Proxy-Enable Web Cache|2|rt=Aug 07 2019 08:47:12 cat=Access Log dst=x ...

in brief, the syslog header is missing. So is it possible to keep the syslog header and save the whole message to file?

[Badger]

By default the syslog input uses a grok pattern that parses the priority, timestamp, host, facility, etc. You could override the grok_pattern option with a pattern that just captures the entire message.

[dickens88 (Charles Liu)]

@Badger
Thank you for your reply. I'm new in logstash, so i'm not sure how to try your suggestion. I tried to add this configuration in logstash.conf, but it doesn't work.

filter {
    grok {
        match => { "message" => "%{GREEDYDATA:message}" }
    }
}

could you pls kindly note the details.tks

[Badger]

Not a grok filter, but the grok_pattern option on the filter. Note the warning there about parsing the timestamp. You will have to add a grok that gets the timestamp out of the message (you can use the same grok pattern that the syslog filter defaults to) and remove the tag that the syslog filter adds.

[dickens88 (Charles Liu)]

@Badger
Accroding to your suggestion, I update my configuration to this. It works then, although I don't know if it is grace... Thank you so much Badger.

input{
     syslog {
        port => 1522
        grok_pattern => "%{GREEDYDATA:message}"
     }
}