Hello Logstash Community,
in my logfiles I have different Exceptions from different Stack Traces. How can I handle singleline and multiline entries within one logfile?
The normal logfile looks like:
<191>Jul 10 13:08:40 FFM30SERVERT0031 DEVTEST2-OO-Logs: Debug,10,metrosystems.mcrm.business.Service.LDAPService,hd0mbbykwvlnidzaxtbquvvc,de,,,LDAP Authentication successful for User 'SUP_A.SCHITZ_US@ASF.ADM.NET'.,,LDAPService,LogEntry
Than I have in the same logfile entries like:
<187>Jul 10 13:09:02 FFM30WEBZST0031 DEVTEST2-OO-Logs: Error,10,MyMetroController,hd0mbbykwvlnidzaxtbquvvc,de,,,Error getting session data for user WoytonGmbH,\"System.ServiceModel.EndpointNotFoundException: There was no endpoint listening at http://id-pp.metrogroup-networking.com/ecommerce/myMetro/webService/ManageUserAccount.asmx that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 10.16.207.143:80
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
at System.Net.Sockets.Socket.InternalConnect(EndPoint remoteEP)
at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Exception& exception)
--- End of inner exception stack trace ---
at System.Net.HttpWebRequest.GetRequestStream(TransportContext& context)
at System.Net.HttpWebRequest.GetRequestStream()
at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream()
--- End of inner exception stack trace ---
Server stack trace:
at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream()
at System.ServiceModel.Channels.HttpOutput.Send(TimeSpan timeout)
at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.SendRequest(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at metrosystems.mcrm.data.DataProvider.Webservice.ManageUserAccount.ManageUserAccountSoap.getAccountByUPN(String UPN)
at metrosystems.mcrm.data.DataProvider.Webservice.LDAPWebserviceDataProvider.GetAccountByUPN(String loginName) in d:\\TeamCity\\buildAgent\\work\\e9218e867e0db194\\metrosystems.mcrm.data\\DataProvider\\Webservice\\LDAPWebserviceDataProvider.cs:line 55
at metrosystems.mcrm.data.Repository.LDAPRepository.GetCardholderByLogin(String loginName) in d:\\TeamCity\\buildAgent\\work\\e9218e867e0db194\\metrosystems.mcrm.data\epository\\LDAPRepository.cs:line 80
at metrosystems.mcrm.oo2.Controllers.MyMetroController.Login(LoginModel loginModel) in d:\\TeamCity\\buildAgent\\work\\e9218e867e0db194\\metrosystems.mcrm.oo2\\Controllers\\MyMetroController.cs:line 213
System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 10.16.207.143:80
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
For the first part I am using the following filter and it works very good:
filter {
if [type] == "online_ordering" {
if !("_grokparsefailure" in [tags]) {
grok {
match => [ "message", "<%{BASE10NUM:}>%{MONTH:month}%{SPACE}%{MONTHDAY:day}%{SPACE}%{TIME:time}%{SPACE}%{HOSTNAME:server}%{SPACE}%{DATA:logfile}:%{SPACE}%{GREEDYDATA:mesag}" ]
remove_field => [ "host" ]
}
csv {
columns => ["level","severity", "logger", "aspnet-sessionid", "organisation", "correlationState", "user", "msg", "exception", "method", "messageType"]
separator => ","
source => "mesag"
remove_field => [ "severity" ]
}
}
}
}
But how I can also handle the lines which additionally throwing Exceptions?
Does anybody have an idea?
Thanks
Greetings,
ABecker