Logstash grok don't parse value

Elastic Stack Logstash
1

Hello, I have the problem that when I try to use grok regex then I can't take others values like LOGLEVEL and so on maybe someone can help me there is my code:

input {
beats {
port => 5043
}
}

filter {
if [host] == "miram.int.bite.lt" {
grok {
match => { "message" => "(?<(.*)</cc.>)"}
match => { "message" => "%{LOGLEVEL:loglevel}"}
}
mutate {
gsub => ["parsedXML", """, "'"]
}

xml {
  source => "parsedXML"
  remove_namespaces => "true"
  xpath => ["//msisdn/text()", "msisdn",
            "//ocsIp/text()", "ocsIP",
            "//sessionId/text()","sessionId",
            "//sgsnIp/text()", "sgsnIp",
            "//ggsnIp/text()", "ggsnIp",
            "//sgsnMccMnc/text()", "sgsnMccMnc",
            "//apn/text()", "apn",
            "//requestType/text()", "requestType"
           ]
  store_xml => "false"
}

}
}

output {
if [host] == "miram.int.bite.lt" {

elasticsearch {
  hosts => ["blablabla:9200"]
  index => "ocs-%{+YYYY.MM.dd}"
}

}
}

And this is my log example:

2018-04-12 11:29:41,035 INFO [net.bitegroup.ocs.lt.sbb.OcsSbb] Request CCR: <?xml version="1.0" encoding="UTF-8" standalone="yes"?>37064195626internpltc13-10-225-64-26-epg02;1516844161;21249996213.226.158.41213.226.158.15424602010151009753584070680628108242f620008042f6200011e60d24602396836756510.20.228.290400rs_lbs_10rgINITIAL_REQUEST000

2

dont show all my log so I add again:

2018-04-12 11:29:41,035 INFO [net.bitegroup.ocs.lt.sbb.OcsSbb] Request CCR: <?xml version="1.0" encoding="UTF-8" standalone="yes"?><ccr><msisdn>37064195626</msisdn><ocsIp></ocsIp><apn>internplt</apn><sessionId>c13-10-225-64-26-epg02;1516844161;21249996</sessionId><sgsnIp>213.226.158.41</sgsnIp><ggsnIp>213.226.158.154</ggsnIp><imsi>246020101510097</imsi><imei>5358407068062810</imei><userLocationInfo>8242f620008042f6200011e60d</userLocationInfo><sgsnMccMnc>24602</sgsnMccMnc><chargingId>3968367565</chargingId><ip>10.20.228.29</ip><qos></qos><chargingCharacteristic>0400</chargingCharacteristic><chargingRuleName>rs_lbs_10rg</chargingRuleName><requestType>INITIAL_REQUEST</requestType><requestNumber>0</requestNumber><creditControlFailureHandlingType>0</creditControlFailureHandlingType><ccSessionFailover>0</ccSessionFailover></ccr>

3

I try to take loglevel and my xml like this:
%{LOGLEVEL:LEVEL}?%{SPACE}(?<parsedXML><(.*)<\/cc.>)

But then I get loglevel null :frowning: Im checking with this tool https://grokdebug.herokuapp.com/

4

Your first attempt fails because the grok filter exits after the first match. Your second attempt fails because the expression just doesn't match your input.

Use a single expression that matches the whole input string. Something like this might work:

^%{TIMESTAMP_ISO8601} %{LOGLEVEL} \[[^\]]+\] Request CCR: %{GREEDYDATA}

I left out the field names.

5

I decide to take xml value with grok match:
match => { "message" => "(?<parsedXML><(.*)<\/cc.>)"}
and then other values take with dissect like this :
dissect { mapping => { "message" => "%{ts} %{+ts} %{loglevel} %{msg}" } }

@magnusbaeck thanks for your answer :slight_smile:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.