Bluecoat proxySG log parsing

Hi, I am facing a problem related to bluecoat proxySG logs;

I have my bluecoat log files stored in a server running an instance of logstash, let's call it server 1. The input section of the configuration file is also on server 1. I set up a second server that runs a second instance of logstash, along with with elasticsearch. Let's call it server 2.

I set up a csv filter to parse my logs, and placed the whole configuration on server 2, but KIBANA doesn't seem to take filter section into account. I am pretty sure my configuration is Okay, because it somehow works when I place it on server 1, but have no idea why it is ignored on server 2. I can see raw logs though, so the problem is not coming from an empty imput.

Maybe someone went through the same situation and could help me to shed light on why the parsing is not working?

Providing your config would be helpful.

All right, this is the input configuration, placed on server 1 (running redis and logstash):

input {
  file {
    path => "/path/to/logs/*.log"
    type => "logstash"
    tags => ["bluecoat"]
    start_position => "beginning"
 }
}

Here is the configuration on server 2 (running elasticserach and logstash):

input {
  redis {
     host => "server1"
     data_type => "list"
     type => "redis-input"
     key => "logstash"
     threads => 4 # number (optional), default: 1
  }
}


filter {
    if [type] == "logstash" {
      if [path] == "/path/to/log/file-name.log" {

               # drop comment lines
               if ([message] =~ /^#/) {
                  drop{}
                      }

    csv {
               columns => ["localtime", "time_taken", "c_ip", "cs_x_forwarded_for", "sc_status", "s_action", "sc_bytes", "cs_bytes", "cs_method", "cs_uri_scheme", "cs_host", "cs_uri_port", "cs_uri_path", "cs_uri_query", "cs_username", "cs_auth_group", "s_hierarchy", "s_supplier_name", "rs_content_type", "cs_referer", "cs_user_agent", "sc_filter_result", "cs_categories", "x_virus_id", "s_ip"]
               separator => " "
               source => message
               remove_field => ["message","host","path","@version","@timestamp"]
          }


if [localtime] {
       date {
           match => ["localtime", "[dd/MMM/YYYY:HH:mm:ss Z]"]
            }
        }

#enrich log entry with destination geolocation info
        if ([c_ip] and [c_ip] != "-" ){
              mutate {
              add_field => {"cs_host_ip" => "%{c_ip}"}
              }

        dns {
             resolve => ["cs_host_ip"]
             action => "replace"
          }

        geoip {
           source => "cs_host_ip"
        }
    }

# parse User-Agent header
if ([cs_user_agent] and [cs_user_agent] != "" and [cs_user_agent] != "-") {
useragent {
source => "cs_user_agent"
prefix => "user_agent_"
}
}


# split Blue web site categories into an array
if ([cs_categories] and [cs_categories] != "" and [cs_categories] != "-") {
mutate {
split => { "cs_categories" => ";" }
}
}


# type convert number fields
mutate {
convert => ["sc_bytes", "integer",
"time_taken", "integer",
"r_port", "integer",
"s_port", "integer",
"cs_bytes", "integer",
"duration", "integer" ]
}

   }
 }
}

I have to mention that bluecoat logs are not the only ones treated by Logstash. I have other logs coming via Syslog, and their parsing works fine.

And here is what a single log line looks like:

"[21/May/2014:17:12:44 +0100]" 18 10.20.2.33 - 200 TCP_HIT 5689 169 GET http www.google.fr 80 / - NavInternet - - 211.85.89.15 pub/sky - - OBSERVED "url_allowed;Search Engines/Portals" - 163.10.165.24

Thanks in advance.

Given

type => "redis-input"

in your redis input configuration, is the

if [type] == "logstash" {

conditional really correct? Use a stdout { codec => rubydebug } to inspect exactly what your events look like when Logstash is done with them.

Yes, in fact there is the "type" parameter that may be wrong in the elasticsearch configuration (not in the redis one as you mentioned), but it doesn't seem to affect the other parsings.

I'll run a debug and give you a feed back as soon as possible.

I changed the configuration. I am actually looking inside the tags array ( the one defined in redis input) for the element "bluecoat" and passing the event to the csv filter if the element is found, like this:

filter {
if "bluecoat" in [tags] {

     # drop comment lines
       if ([message] =~ /^#/) {
            drop{}
             }
...

Nothing's changed, I still get no results.

I even tried with Grok:

filter {
if "bluecoat" in [tags] {

# drop comment lines
if ([message] =~ /^#/) {
drop{}
}

grok {
match => ["message","\"\[%{DAY}/%{MONTH}/%{YEAR}:%{TIME} %{ISO8601_TIMEZONE}\]\" %{NUMBER:time_taken} %{IP:c_ip} %{IP:cs_x_forwarded_for} %{NUMBER:status} %{GREEDYDATA:s_action} %{NUMBER:sc_bytes} %{NUMBER:cs_bytes} %{WORD:cs_method} %{URIPROTO:cs_uri_scheme} %{IPORHOST:cs_host} %{NUMBER:cs_uri_port} %{URIPATH:cs_uri_path} %{URI:cs_uri_query} %{USERNAME:cs_username} %{GREEDYDATA:cs_auth_group} %{GREEDYDATA:s_hierarchy} %{GREEDYDATA:s_supplier_name} %{GREEDYDATA:rs_content_type} %{GREEDYDATA:cs_referer} %{GREEDYDATA:cs_user_agent} %{GREEDYDATA:sc_filter_result} %{GREEDYDATA:cs_categories} %{GREEDYDATA:x_virus_id} %{IP:s_ip}"]

add_tag => [ "grokked_bluecoat" ]
tag_on_failure => [ "ERROR : grok matching failed" ]
}

Same thing. I tried to run a debug, I get a java buffer over flow exception...
Here is the command I typed:

/path/to/logstash --debug -l /path/to/log_file -f /path/to/config/

Here is the exception I get:

HeapByteBuffer.java:183:in `put': java.nio.BufferOverflowException

The whole configuration above, from the beggining, is on server 2
Does it ring any bell?

HeapByteBuffer.java:183:in `put': java.nio.BufferOverflowException

Nothing I've seen before. Do you have the full stack trace?

Again, getting the result of a stdout { codec => rubydebug } output for a message that isn't correctly filtered would be helpful.

The stack is too big and can't fit in here, here is the first part of it:

Exception in thread "[main]>worker2" Exception in thread "[main]>worker1" java.nio.BufferOverflowException
        at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:183)
        at org.jruby.util.io.ChannelStream.bufferedWrite(ChannelStream.java:1100)
        at org.jruby.util.io.ChannelStream.fwrite(ChannelStream.java:1277)
        at org.jruby.RubyIO.fwrite(RubyIO.java:1541)
        at org.jruby.RubyIO.write(RubyIO.java:1412)
        at org.jruby.RubyIO$INVOKER$i$1$0$write.call(RubyIO$INVOKER$i$1$0$write.gen)
        at org.jruby.RubyClass.finvoke(RubyClass.java:743)
        at org.jruby.runtime.Helpers.invoke(Helpers.java:505)
        at org.jruby.RubyBasicObject.callMethod(RubyBasicObject.java:364)
        at org.jruby.RubyIO.write(RubyIO.java:2490)
        at org.jruby.RubyIO.putsSingle(RubyIO.java:2478)
        at org.jruby.RubyIO.puts1(RubyIO.java:2407)
        at org.jruby.RubyIO.puts(RubyIO.java:2380)
        at org.jruby.RubyIO$INVOKER$i$puts.call(RubyIO$INVOKER$i$puts.gen)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)
        at rubyjit.Cabin::Outputs::IO$$\=\^\^_a759d2d46347217e5f0d618914b90c91265cea161954102985.block_0$RUBY$__file__(/path/to/logstash/vendor/local_gems/54c50e3f/cabin-0.8.1/lib/cabin/outputs/io.rb:52)
        at rubyjit$Cabin::Outputs::IO$$\=\^\^_a759d2d46347217e5f0d618914b90c91265cea161954102985$block_0$RUBY$__file__.call(rubyjit$Cabin::Outputs::IO$$\=\^\^_a759d2d46347217e5f0d618914b90c91265cea161954102985$block_0$RUBY$__file__)
        at org.jruby.runtime.CompiledBlock19.yield(CompiledBlock19.java:135)
        at org.jruby.runtime.Block.yield(Block.java:142)
        at org.jruby.ext.thread.Mutex.synchronize(Mutex.java:149)
        at org.jruby.ext.thread.Mutex$INVOKER$i$0$0$synchronize.call(Mutex$INVOKER$i$0$0$synchronize.gen)
        at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:143)
        at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:154)
        at rubyjit.Cabin::Outputs::IO$$\=\^\^_a759d2d46347217e5f0d618914b90c91265cea161954102985.__file__(/path/to/logstash/vendor/local_gems/54c50e3f/cabin-0.8.1/lib/cabin/outputs/io.rb:50)
        at rubyjit.Cabin::Outputs::IO$$\=\^\^_a759d2d46347217e5f0d618914b90c91265cea161954102985.__file__(/path/to/logstash/vendor/local_gems/54c50e3f/cabin-0.8.1/lib/cabin/outputs/io.rb)
        at org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:181)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)
        at org.jruby.runtime.callsite.ShiftLeftCallSite.call(ShiftLeftCallSite.java:24)
        at rubyjit.Cabin::Subscriber$$\=\^\^_10bd20586d047ca25f2f05ebf603ebf4af74e6dd1954102985.__file__(/path/to/logstash/vendor/local_gems/54c50e3f/cabin-0.8.1/lib/cabin/subscriber.rb:10)
        at rubyjit.Cabin::Subscriber$$\=\^\^_10bd20586d047ca25f2f05ebf603ebf4af74e6dd1954102985.__file__(/path/to/logstash/vendor/local_gems/54c50e3f/cabin-0.8.1/lib/cabin/subscriber.rb)
        at org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:181)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)
        at org.jruby.runtime.callsite.ShiftLeftCallSite.call(ShiftLeftCallSite.java:24)
        at rubyjit.Cabin::Channel$$publish_2fedc2a9a4a36f3ad01bf44594d627e37af6b87f1954102985.block_2$RUBY$__file__(/path/to/logstash/vendor/local_gems/54c50e3f/cabin-0.8.1/lib/cabin/channel.rb:196)

The parsing works fine now. I have my csv bluecoat fields on KIBANA thanks to my latest configuration. ALthough the fields are not recongnized on dashboard visualizations, is it normal? They are tagged with a question mark on the discover KIBANA page.

Problem solved after reloading the parameters, in the settings section. Thank you all for your help.

If you find something on the java stack error, this would help, thanks again.

1 Like