Parsing json logs issue

Hi All,

I am rather new to all of this and I have been tasked to get logging working in our environment. We have a smoothwall firewall that we use for webfiltering. I have manged to get most of the logs parsed, however I would like to get the username and this is where I am getting stuck.

</> "tagset":{"Protocol":{"HTTPS":[]},"auth":{"finished":[]},"authmethod":{"core":[]},"group":{"9":[]},"localip":{"x.x.x.x":[]},"localport":{"xxx":[]},"rurlcategory":{"Connect for Chromebooks":["^https?:\/\/(?!.*?encrypted-v?tbn\\d).*?\\.?gstatic\\.com"]},"tenant":{"numbers":[]},"urlcategory":{"Connect for Chromebooks":[""],"Content Delivery":[""]},"username":{"domain\\user":[]}}

I have created the following filter

</> filter {

  if [type] == "squid" 


      json {

          source => "message"


      date {

          match => [ "time","UNIX" ]

          target => "@timestamp"

          remove_field => [ "time" ]


      split {

        field => "ruleid"

        field => "tagset"


      mutate {

            add_field => {

              "username" => "%{[tagset][username]}"


            remove_field => ["message", "@timestamp", "tenant", "took"]




The username that shows in Kibana is displayed as {"domain\username":[]}
Any Ideas on what I may need to do to get the username to show as domain\username?

I am surprised logstash will even start if you give it that configuration. The two instances of the option are combined to form an array, and the field option of a split filter wants a string (field name), not an array. You may have meant

  split { field => "ruleid" }
  split { field => "tagset" }

but neither field is an array, so both split filters will fail.

To extract the username you could use a ruby filter. My apologies if my ruby code makes your eyes bleed, but the following does work

    ruby {
        code => '
            u = ""
            event.get("[tagset][username]").each { |k,v| u = k }
            event.set("username", u)

You are correct. Logstash failed to start, so I removed those lines of code. My apologies for leaving it in this post.

Thank you for the ruby code. I am new to all of this, so the code looks very confusing to me anyway. That worked like a charm. Thank you so much for your help with this.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.