LogStash config not parsing out log events even though config checks out in Grok Debugger

[cchooks2 (Steven)]

First post and hoping this is the forum to post questions ...

I have the following GROK filter:

if ([service] == "MY_SERVICE"){
if([attributes][file_path] == "/apps/log/mydoc/wso2carbon.log") {
grok {
patterns_dir => [ "/apps/logstash-patterns" ]
match => [ "body", "%{wso2carbon}" ]
}
kv {
source => "kvpairs"
field_split => ", "
value_split => " = "
remove_field => [ "kvpairs" ]
}
date {
match => ["ts","UNIX_MS"]
target => "@timestamp"
timezone => "America/New_York"
}
}
}

GROK Patterns:

CUSDATE [0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2},[0-9]{3}
TYPE [A-z\s]+
wso2carbon %{GREEDYDATA:tid}%{SPACE}[%{CUSDATE:timestamp}]%{SPACE}[%{GREEDYDATA:message}]%{SPACE}[%{GREEDYDATA:req_id}]%{SPACE}%{WORD:loglevel}%{SPACE}{%{GREEDYDATA:class}}%{SPACE}-%{SPACE}%{TYPE:type}:%{GREEDYDATA:kvpairs}

Log I am trying to filter:

TID: [0] [AM] [2017-08-10 13:35:24,157] [PassThroughMessageProcessor-264] [FSREQID=sdasfdfsrgrwgfdfsafda] INFO {some_url_here} - Initiating Request : ClientIp = 10.xx.x.xx, ClientHost = myhost, xClientIp = 10.xxx.xx.xxx, xForwardedFor = null, CorrelationID = xxxxxxxxddsdfcdsfewewdef, FSREQID = dsaefr24ffewefefre, HTTPMethod = GET, Url = some/url/here, ContentType = null

I am testing my Grok filter using the GROK debugger and everything checks out.

When I start up my LogStash it is not throwing any errors about the config. When i query the data in Kibana I do not see the _grokparsefailure on the data.

Not sure what is going on here any help would be appreciated.

Thanks!

[magnusbaeck (Magnus Bäck)]

Show what a raw event processed by Logstash looks like. Use a stdout { codec => rubydebug } output.

[cchooks2 (Steven)]

I was having issues getting it to print out on the terminal .... i had to change the regex that was parsing out the timestamp to {TIMESTAMP_ISO8601:timestamp} and that actually parsed out to the console. I also changed the date stanza to:

date {
match => ["timestamp" ,"yyyy-MM-dd HH:mm:ss,SSS"]
target => "@timestamp"
timezone => "America/New_York"
}

timestamp: "2017-08-11 17:10:22,212

Below you can see the output, but unfortunately it is still not showing up in Kibana parsed out. No grokparsefailure either.

       "path" => "/apps/logstash-5.2.0/bin/test3.txt",
 "@timestamp" => 2017-08-11T21:20:31.586Z,
   "@version" => "1",
       "host" => "myhost",
  "log_level" => "WARN",
"log_message" => "__SynapseService Executing fault sequence mediator : fault",
    "message" => "2017-08-11 17:10:22,212 [-] [PassThroughMessageProcessor-280] []  WARN __SynapseService Executing fault sequence mediator : fault",
   "messageB" => "PassThroughMessageProcessor-280",
  "timestamp" => "2017-08-11 17:10:22,212",
   "messageA" => "-"

Actual message:
2017-08-11 17:10:22,212 [-] [PassThroughMessageProcessor-280] [] WARN __SynapseService Executing fault sequence mediator : fault

[magnusbaeck (Magnus Bäck)]

Where are the service and [attributes][file_path] fields? You only apply the grok filter if those fields have certain values. But some grok filter or similar is obviously being used since you have fields like log_level and log_message.

[cchooks2 (Steven)]

For this particular test to the terminal I am not invoking the service and attributes field paths ... I am testing the GROK filter directly to the event I am trying to parse. The events is getting parsed out correctly.

The service and attributes field path are specific to each of the events i want parsed out. The service will be the same for the four type of events i want parsed out, but the file path will be specific to the event.

For example:

else if ([service] == "my_service"){
if([attributes][file_path] == "/apps/log/my_service/wso2-errors.log") {

else if ([service] == "my_service"){
if([attributes][file_path] == "/apps/log/my_service/wso2-service.log") {

else if ([service] == "my_service"){
if([attributes][file_path] == "/apps/log/my_service/gc.log") {

else if ([service] == "my_service"){
if([attributes][file_path] == "/apps/log/my_service/wso2carbon.log") {

[cchooks2 (Steven)]

Figured this out. I was calling the same service in my else ifs. I just nested if's within the else and add the corresponding paths.