How to parse bluecoat proxyAV logs

Elastic Stack Logstash
1

Format string from Bluecoat ProxyAV
c-dns, -, -, N, date, time, 1, -, -, s-dns, s-ip, s-port, time-taken, sc-bytes, cs-bytes, cs-protocol, -, cs-method, cs-uri, sc-mimetype, cache, sc-status

Output in the logstash message
192.1.1.1, -, -, N, 2015-10-02, 13:23:22, 1, -, -, ProxyAV01, 192.2.1.1, 1344, 31500, 224, 0, ICAP, -, RESPMOD, http://www.cnet.com, -, Inet, 204

How can i parse the message into a more searchable and readable fields into logstash?

2

This is a good match for the csv filter. The columns containing numerical data won't be converted to integer fields so you'll want to do that afterwards with a mutate filter.

Make sure you understand what happens if a field value contains a comma. This can e.g. happen with the cs-uri field. If you set the csv filter's separator to ", " (comma space) you should be fine but I'd make sure there aren't any corner cases.

3

is this the way to do it?

filter {

csv {

columns => ["c-dns", "date", "time", "s-dns", "s-ip", "s-port", "time-taken", "sc-bytes", "cs-bytes", "cs-protocol", "cs-method", "cs-uri", "sc-mimetype", "cache", "sc-status"]
separator => ","

}
grok {
        match => ["message","%{IP:c-dns} %{DATESTAMP:date} %{DATESTAMP:time} %{WORD:s-dns} %{IP:s-ip} %{NUMBER:s-port} %{NUMBER:time-taken} %{NUMBER:sc-bytes} %{NUMBER:cs-bytes} %{WORD:cs-protocoal} %{WORD:cs-method} %{URIPATHPARAM:cs-uril} %{WORD:cs-mimetype} %{WORD:cache} %{NUMBER:sc-status}"]
}

}

4

Well, why don't you try it out?

But drop the grok filter. It duplicates the job of the csv filter (plus the expression won't match anything since you've omitted the commas). Also, as I suggested, make the csv filter's separator ", " (comma space) instead of just a comma.

Or, use the grok filter instead of the csv filter. Properly configured either will work. The benefit of the grok filter is that it's easier to deal with corner cases and you can capture integer fields right away without needing a follow-up mutate filter to convert some of the strings to integers.