Unable to create logstash pattern using grok for below


(Hanit Grover) #1

INFO 2018-09-26 18:55:10,802 [[logging_framework].HTTP_Listener_Configuration.worker.01] com.temp.analytics.Logger: {"flowLevel":"subflow","apiName":"test_MainFlow","transactionId":"df782540-c169-11e8-8c83-40b89aa3f91a","sourceSystemName":"MMO","sourceSystemIP":"null","sourceSystemURL":"/test/account","HTTPHeaders":"{http.version=HTTP/1.1, accept-encoding=gzip, deflate, http.remote.address=/127.0.0.1:61393, connection=keep-alive, http.uri.params=ParameterMap{[]}, sourcesystemurl=/test/account, postman-token=0c501fd3-fc7f-4cb5-a65e-1f93812682bc, http.relative.path=/testLogger, http.listener.path=/testLogger, http.request.uri=/testLogger, http.method=GET, cache-control=no-cache, targetsystemname=janrain, http.query.params=ParameterMap{[]}, host=localhost:8081, user-agent=PostmanRuntime/6.1.6, sourcesystemname=MMO, accept=/, http.scheme=http, http.request.path=/testLogger, http.query.string=}","resource":"/testLogger","HTTPOperation":"GET","applicationName":"logging_framework","applicationURI":"/testLogger","datetimeIN":"1537952110142","targetSystemName":"janrain","dateTimeOut":"1537952110189","httpStatus":"null","errorMessage":"null","timeElapsed":"632"}

Output fields to be :slight_smile:"flowLevel" : { "type" : "keyword" },
"transactionId" : { "type" : "text" },
"sourceSystemName" : { "type" : "keyword" },
"sourceSystemIP" : { "type" : "ip" },
"sourceSystemURL" : { "type" : "text" },
"httpHeaders" : { "type" : "text" },
"resource" : { "type" : "keyword" },
"httpOperation" : { "type" : "keyword" },
"applicationName " : { "type" : "keyword" },
"applicationURI" : { "type" : "keyword" },
"dateTimeIn" : { "type" : "date" },
"targetSystemName" : { "type" : "text" },
"dateTimeOut" : { "type" : "date" },
"httpStatus" : { "type" : "keyword" },
"errorMessage" : { "type" : "text" },
"timeElapsed" : { "type" : "text" },
"logEvenSentTime" : { "type" : "date" },
"apiName" : { "type" : "keyword" }


(Andreas H) #2

This is a great tool for you to create a GROK pattern to match what you want:
https://grokdebug.herokuapp.com/
Give it a shot and maybe post your GROK pattern after you've tried it yourself first and then someone can help you.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.