Extract specifc field from greedydata

Hi Team,
I am working on specific set of logs and it follows simple grok structure as shown below.


the "LOGMESSAGE" field does contain some user related information at times such as userId & SessionId. I have now been given a requirement to extract those fields from LOGMESSAGE wherever they are present, is there a way to achieve it !!

You are asking us how to parse a field that matches GREEDYDATA (i.e. anything)? You need to tell us what that field looks like, and what pattern you want to parse from it.

ok here some of the examples of greedydata.

POST /current/v1/advisors/codes "-" "" "HTTP/1.1" XXXXXXXX "-" "-"
GET /current/v1/user/mrdrysdale64/current "-" "?sessionId=3c786e4c-99d6-b15a-4658-cb7016552289&transactionId=29aa9d09-0b81-d6fb-ce5e-e795d031826d" "HTTP/1.1" XXXXXXXX "-" "-"
GET /current/v1/user/mrdrysdale64/current "-" "?sessionId=90e01589-25fb-526b-ce9f-261e87d70128&transactionId=765153d3-861a-acfc-ab3f-1629800eeda9" "HTTP/1.1" XXXXXXXX
POST /current/v1/pattern/codes "-" "" "HTTP/1.1" XXXXXXXXX "-" "-"
GET /current/v1/user/grzeg1tl/current "-" "?sessionId=291dd1fd-3e64-69a8-2450-af0706768c44&transactionId=93be0267-e84c-f002-e7d8-3f54ebdfd7df" "HTTP/1.1" XXXXXXXX "-" "-
POST /current/v1/authenticate "-" "?sessionId=63e04384-c17c-ce50-9c44-84a99f15e6bd&transactionId=f46453bd-017b-89e6-b05f-a8fc356eaefe" "HTTP/1.1" XXXXXXXXX "-" "-"

Since all of the messages are not consitent, i parsed it has greedydata field, but now requirement is to extract sessionId and transactionId from the above messaged(wherever available), and if possible the username field which is available after "user/" in the above messages.

grok { match => { "message" => '^%{NOTSPACE} %{NOTSPACE:[@metadata][uri]} "%{DATA}" "%{NOTSPACE:[@metadata][ids]}" ' } }
if [@metadata][uri] =~ /\/user\/[^\/]+\// {
    grok { match => { "[@metadata][uri]" => "\/user\/(?<userid>[^/]+)\/" } }
kv { source => "[@metadata][ids]" field_split => "&?" value_split => "=" }

Note that different parsers consume the regexp in the conditional and the regexp in the grok, so they need different escaping.

Thanks Badger for your big help, while i am able to ectract the id fields as expected i have also been asked to give sudo display name for the "transactionId" field in the above message. Can it be done inside the kv script!!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.