Grokfilter for JSON Field not working

Elastic Stack Logstash
1

Hello,

i'm a beginner of Logstash and Grok so please tell me if i have missunderstood a basic concept, there is no stupid answer even if it sounds too easy and obvious :slight_smile:

I got a working config for IIS Logs and WindowsEventLogs but i'm breaking my brain getting an Log4Net Application log to get parsed into elastic. Our Devs using an old version of Log4Net which don't have the ability to log into JSon (log4net extension) so i have to parse the multiline textfile.

A Sample:

2015-04-23 10:05:04,838 ERROR Framework.Application.Implementation.ApplicationImpl - Call to: ConvertPdf failed; System.Exception: ConvertPdf failed. ---> System.ArgumentOutOfRangeException: Der Index lag auà ¥rhalb des Bereichs. Er darf nicht negativ und kleiner als die Auflistung sein. Parametername: index
bei System.Collections.Generic.List`1.get_Item(Int32 index)
bei Framework.Application.Implementation.Helper.PdfConverter.ConvertToPng(Stream output, Int32 resolution, Int32 pageNum, Single scale, Int32 rotation) in c:\TeamCity\buildAgent\work\c073c09c50e2e05b\Framework.Application.Implementation\Helper\PdfConverter.cs:Zeile 44.
bei Framework.Application.Implementation.ApplicationImpl.ConvertPdf(Byte buffer, Int32 pageNum, Int32 resolution, Double scale, Int32 rotAngle, String password) in c:\TeamCity\buildAgent\work\c073c09c50e2e05b\Framework.Application.Implementation\ApplicationImpl.cs:Zeile 40.
--- Ende der internen Ausnahmestapel𢥲wachung ---
bei Framework.Application.Implementation.ApplicationImpl.ConvertPdf(Byte buffer, Int32 pageNum, Int32 resolution, Double scale, Int32 rotAngle, String password) in c:\TeamCity\buildAgent\work\c073c09c50e2e05b\Framework.Application.Implementation\ApplicationImpl.cs:Zeile 58.
bei SyncInvokeConvertPdf(Object , Object , Object )
bei System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object inputs, Object& outputs)
bei System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
bei System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)
bei System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)
bei System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)
2015-04-28 11:06:06,002 INFO Framework.Application.Host.ApplicationHost - Service stopped

My NXLog Config for those logs seems to work fine:

InputType multiline Module im_file File "L:\\INVOICETRACK\\C3T\\Log\\Springtime.ConverterService\\applog_*" SavePos FALSE Exec $Hostname = hostname_fqdn(); Exec $Environment = "C3T"; Exec $Application = "ConverterService"; Exec $logmessage = $raw_event; Exec to_json();
Module om_tcp Host 192.168.50.171 Port 3334

<Route 3>
Path ConverterServiceLogs => converterServiceOut

2

Part2:

The output from nxlog looks something like:

{"EventReceivedTime":"2016-07-29 08:35:31","SourceModuleName":"ApplicationLogs","SourceModuleType":"im_file","Hostname":"some-hostname.domain.local","Environment":"C3T","Application":"Application","logmessage":"2015-04-23 10:05:04,838 ERROR Framework.Application.Implementation.ApplicationImpl - Call to: ConvertPdf failed; System.Exception: ConvertPdf failed. ---> System.ArgumentOutOfRangeException: Der Index lag auࠥrhalb des Bereichs. Er darf nicht negativ und kleiner als die Auflistung sein. Parametername: index\r\n bei System.Collections.Generic.List`1.get_Item(Int32 index)\r\n bei Framework.Application.Implementation.Helper.PdfConverter.ConvertToPng(Stream output, Int32 resolution, Int32 pageNum, Single scale, Int32 rotation) in c:\TeamCity\buildAgent\work\c073c09c50e2e05b\Framework.Application.Implementation\Helper\PdfConverter.cs:Zeile 44.\r\n bei Framework.Application.Implementation.ApplicationImpl.ConvertPdf(Byte buffer, Int32 pageNum, Int32 resolution, Double scale, Int32 rotAngle, String password) in c:\TeamCity\buildAgent\work\c073c09c50e2e05b\Framework.Application.Implementation\ApplicationImpl.cs:Zeile 40.\r\n --- Ende der internen Ausnahmestapel𢥲wachung ---\r\n bei Framework.Application.Implementation.ApplicationImpl.ConvertPdf(Byte buffer, Int32 pageNum, Int32 resolution, Double scale, Int32 rotAngle, String password) in c:\TeamCity\buildAgent\work\c073c09c50e2e05b\Framework.Application.Implementation\ApplicationImpl.cs:Zeile 58.\r\n bei SyncInvokeConvertPdf(Object , Object , Object )\r\n bei System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object inputs, Object& outputs)\r\n bei System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)\r\n bei System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)\r\n bei System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)\r\n bei System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet) "}

Also the Grok-Pattern works when i test it with http://grokconstructor.appspot.com/

Logstash -Input:

input {
tcp {
type => "ApplicatonLogs"
port => 3334
codec => json
}
}

filter {
if "Application" == "ConverterService" {

            multiline {
                    pattern => "%{DATESTAMP}"
                    what => "previous"
                    negate => true
            }
            grok {
                    add_tag => "beforeGrokked"
                    match => [ "logmessage","%{DATESTAMP:EventTime} %{LOGLEVEL:Severity} +%{NOTSPACE:class} +%{NOTSPACE:Separator01} (?<text>(.|\r|\n)*)" ]
                    add_tag    => "afterGrokked"
            }
    }

}

But in the Results (got from Kibana/Elasticsearch) i do not see any parsed fields only the "logmessage" i tried to parse.

Result:

{
"_index": "applicationlogs",
"_type": "ApplicatonLogs",
"_id": "AVY1XXLLqb4zekNy2EbM",
"_score": 1,
"_source": {
"EventReceivedTime": "2016-07-29 08:35:31",
"SourceModuleName": "ConverterServiceLogs",
"SourceModuleType": "im_file",
"Hostname": "webhost.some.domain.local",
"Environment": "C3T",
"Application": "ConverterService",
"logmessage": "2015-04-22 11:52:26,093 INFO Framework.Application.Host.ApplicationHost - Framework Case Service: Initializing.",
"@version": "1",
"@timestamp": "2016-07-29T06:35:31.293Z",
"host": "192.168.51.167",
"port": 50691,
"type": "ApplicatonLogs"
},
"fields": {
"@timestamp": [
1469774131293
]
}
}

is it even possible to grok json value fields in logstash?

3

just saw that i posted the wrong json result,

this is like a multiline json result logmessage looks like:

"logmessage": "2015-04-23 10:05:04,838 ERROR Framework.Application.Implementation.ApplicationImpl - Call to: ConvertPdf failed; System.Exception: ConvertPdf failed. ---> System.ArgumentOutOfRangeException: Der Index lag auࠥrhalb des Bereichs. Er darf nicht negativ und kleiner als die Auflistung sein. Parametername: index\r\n bei System.Collections.Generic.List`1.get_Item(Int32 index)\r\n bei Framework.Application.Implementation.Helper.PdfConverter.ConvertToPng(Stream output, Int32 resolution, Int32 pageNum, Single scale, Int32 rotation) in c:\TeamCity\buildAgent\work\c073c09c50e2e05b\Framework.Application.Implementation\Helper\PdfConverter.cs:Zeile 44.\r\n bei Framework.Application.Implementation.ApplicationImpl.ConvertPdf(Byte buffer, Int32 pageNum, Int32 resolution, Double scale, Int32 rotAngle, String password) in c:\TeamCity\buildAgent\work\c073c09c50e2e05b\Framework.Application.Implementation\ApplicationImpl.cs:Zeile 40.\r\n --- Ende der internen Ausnahmestapel𢥲wachung ---\r\n bei Framework.Application.Implementation.ApplicationImpl.ConvertPdf(Byte buffer, Int32 pageNum, Int32 resolution, Double scale, Int32 rotAngle, String password) in c:\TeamCity\buildAgent\work\c073c09c50e2e05b\Framework.Application.Implementation\ApplicationImpl.cs:Zeile 58.\r\n bei SyncInvokeConvertPdf(Object , Object , Object )\r\n bei System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object inputs, Object& outputs)\r\n bei System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)\r\n bei System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)\r\n bei System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)\r\n bei System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet) ",