Logstash find pattern help

[rohitnrl (rohit)]

Hi,

I have a log as following

2018-05-07 11:38:56,748 [15] ERROR Library.Logger.Log4NetLogger [(null)] - {"Component":"Service","Message":"Exception in AuthorizationBll.CheckAuthorization.Exception:No Roles were found for the login Id: Domain\user0 ","Exception":{"ClassName":"System.Exception","Message":"No Roles were found for the login Id: Domain\User0","Data":{},"InnerException":null,"HelpURL":null,"StackTraceString":" at BusinessLogic.AuthorizationBll.GetRoles(String loginId) in \Services\Services.BusinessLogic\AuthorizationBll.cs:line 146\r\n at Services.BusinessLogic.AuthorizationBll.CheckAuthorization(String loginId, String operationName) in \Services\Services.BusinessLogic\AuthorizationBll.cs:line 61","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":"8\nGetRoles\nServices.BusinessLogic, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null\nServices.BusinessLogic.AuthorizationBll\nSystem.Collections.Generic.List`1[System.String] GetRoles(System.String)","HResult":-2146233088,"Source":"Services.BusinessLogic","WatsonBuckets":null},"ErrorLevel":1,"ErrorLevelType":"Error","CurrentDateTime":"2018-05-07T11:38:56.6918708+05:30","ManagedThreadId":15,"AdditonalData":null,"Context":null}

and I using the Gork Pattern as :

%{SYSLOG5424SD} - {%{QS}:%{QS},%{QS}:%{QS},%{QS}:{%{QS}:%{QS},%{QS}:%{QS},%{QS}:{},%{QS}:null,%{QS}:null,%{QS}:%{QS},%{QS}:null,%{QS}:0,%{QS}:%{QS},%{QS}:%{ISO8601_TIMEZONE}233088,%{QS}:%{QS},%{QS}:null},%{QS}:1,%{QS}:%{QS},%{QS}:%{QS},%{QS}

I want 3 piece of information from this :

1. Component Name and its value
2. Message and its value
   3 Stack trace and value

Can someone kindly help

[magnusbaeck (Magnus Bäck)]

Don't use grok to parse a JSON string. Use a json filter, but first have a grok (or dissect) filter that extracts the timestamp, loglevel, logger name, and finally the JSON string at the end to separate fields.

[rohitnrl (rohit)]

Is there any sample you can share..

if [message] =~ "\A{.+}\z" {
json {
source => "message"
}

    }

    # Some non-json messages are now tagged with _jsonparsefailure. We dont care.
    #mutate { remove_tag => [ "_jsonparsefailure" ] }

this is the pattern im using but the problem is how should i extract data fields from within the message attribute

[magnusbaeck (Magnus Bäck)]

Is there any sample you can share..

There are lots of grok filter examples out there, and the grok constructor web site can help you construct an expression. You only need to extract about a handful of fields from the log message, where the last field is the entire end of the string (containing the JSON message).