Using grok filter to parse log file

Elastic Stack Logstash
#1

Hi guys,

i'm trying to parse the following log files, i succeed to filter all fields unless the last one, can some one help pls, thank you.

Log file:

1;2;06-19-15;start inbound processing;50034744;1;ok;2021-03-18 16:10:44.63; ; ;/var/webMethods/bbb/cbbb/download/06-19-99/xcdc.xml;"FilePolling.60088
"

using Grok Filter:

%{NUMBER:Row Nr};%{DATA:Nr};%{DATA:corrier Nr};%{DATA:was Passiert};%{NUMBER:ID Prozess};%{NUMBER:Indicators};%{DATA:Status};%{TIMESTAMP_ISO8601:Datum};%{SPACE};%{SPACE};%{URIPATHPARAM:location}; /"%{GREEDYDATA:response} /n"

result :slight_smile:

image
image1178×544 21 KB

#2

Why the / in front of the final variable?

#3

hi , to escape the (" ... " ) and the lline after the last variable ans extract onle that Word (FilePolling.60088) .

........; "FilePolling.60088
"

#4

Hi Bab,

Out side of your question, the logs seems to follow a pretty standard delimiter of semicolon ";",
I wonder why don't you use dissect with ; as delimiter instead?

Back to your question, I tested and modified your query in a GROK tester by adding "\s" in before double quote, it seems to work.

%{DATA:Nr};%{DATA:corrier Nr};%{DATA:was Passiert};%{NUMBER:ID Prozess};%{NUMBER:Indicators};%{DATA:Status};%{TIMESTAMP_ISO8601:Datum};%{SPACE};%{SPACE};%{URIPATHPARAM:location};"%{GREEDYDATA:response}\s"

Result

{
  "Nr": [
    "1"
  ],
  "corrier": [
    "2"
  ],
  "was": [
    "06-19-15;start inbound processing"
  ],
  "ID": [
    "50034744"
  ],
  "BASE10NUM": [
    "50034744",
    "1"
  ],
  "Indicators": [
    "1"
  ],
  "Status": [
    "ok"
  ],
  "Datum": [
    "2021-03-18 16:10:44.63"
  ],
  "YEAR": [
    "2021"
  ],
  "MONTHNUM": [
    "03"
  ],
  "MONTHDAY": [
    "18"
  ],
  "HOUR": [
    "16",
    null
  ],
  "MINUTE": [
    "10",
    null
  ],
  "SECOND": [
    "44.63"
  ],
  "ISO8601_TIMEZONE": [
    null
  ],
  "SPACE": [
    " ",
    " "
  ],
  "location": [
    "/var/webMethods/bbb/cbbb/download/06-19-99/xcdc.xml"
  ],
  "URIPATH": [
    "/var/webMethods/bbb/cbbb/download/06-19-99/xcdc.xml"
  ],
  "URIPARAM": [
    null
  ],
  "response": [
    "FilePolling.60088"
  ]
}

Hope this can help you!