Grok failure even after the fields were filtered out successfully

Elastic Stack Logstash
1

Hi Everyone,

I am encountering the following problem: I have been filtering out the "Via_IP1" field from "Via1" which matches on both grok debugger and grok constructor.But when running on logstash ,i could observe grokparsefailure in my tag.

i am using below grok filter ,

   grok{
         id => "grok5"
         match => {"via1" => "^(?:.*SIP/2.0/UDP)(?<Via_IP1>[\s]\d+.\d+.\d+.\d+)?(?:.*)?$" }
         }

Output:

"tags" => [
[0] "multiline",
[1] "_grokparsefailure"
],
"SipFromUser:" => "2002005498",
    "path" => "/var/log/prod_logs/XS/basatlxs01/XS_Invite.txt",
  "@timestamp" => 2017-12-07T14:26:42.313Z,
"channel:" => "Sip ",
    "via2" => " tag=0ee7cb75^M\nCall-ID: sAP9-qj5ZJVY4xLJ3f2DTg..^M\n",
    "via1" => "SIP/2.0^M\nVia: SIP/2.0/UDP 5060;branch=z9hG4bK-524287-1---d0be5d0eb41a1b32;rport^M\nVia: SIP/2.0/UDP 96.118.191.49:5060;branch=z9hG4bK-524287-1---d0be5d0eb41a1b32;rport^M\nMax-Forwards: 70^M\nContact: <sip:2002005498@96.118.191.49:5060>^M\n",
  "ServerName" => "basatlxs01",
"@version" => "1",
    "host" => "prod-platform",
  "Sip_Method" => "INVITE^M",
 "Via_IP1" => " 96.118.000.49"

Could you help me in understating and debugging on this grokparsefailure ?

2

I suspect you have more than one grok filter in your configuration. Perhaps an extra file in /etc/logstash/conf.d that you've forgotten about?

3

Hi Magnus ,

We are using multiple grok in the configuration to filter out the fields.

     grok{
         id => "grok5"
          match => {"via1" => "^(?:.*SIP/2.0/UDP)(?<Via_IP1>[\s]\d+.\d+.\d+.\d+)?(?:.*)?$" }
         }
          grok{
                id => "grok6"
          match => {"via2" => "^(?:.*SIP/2.0/UDP)(?<Via_IP2>[\s]\d+.\d+.\d+.\d+)?(?:.*)?$" }
         }

                   grok {
                id => "grok7"
       match => { "cid" => "(?<correlationid:>callhalf-\d+:\d{1})?$" }
       }
                   grok{
                id => "grok8"
       match => { "path" => "(?:[\w\W]*\/)(?<ServerName>\w+)(?:\/[\w\W]*)" }
       }

is that a problem ?

when we commented out the grok specified above(grok5 and grok6) grokfailure doesn't occur.

We are using command line to run the logstash filter and explicitly specifying the conf path like

sudo bin/ -f /home/conf_file/logstash_xslog_updated_test.conf ...

4

is that a problem ?

No.

when we commented out the grok specified above(grok5 and grok6) grokfailure doesn't occur.

Well, that's a pretty good clue. If we look at the grok6 filter it expects the field to contain "SIP/2.0/UDP" but the via2 field doesn't contain that string.

5

Thanks Magnus:grinning:
We could solve the issue.