Field data from match results

Elastic Stack Logstash
1

I have following that matches incoming rabbit mq data,
match=> { "full_message" => "(?[^;]*);[;]%{NUMBER:payloadSize};[;]%{GREEDYDATA:message_text}" }

My requirement is to extract more fields out of the message text that i am retrieving, for ex, Tag1.Tag2.Tag3 might appear at the begining of the message_text but they can be missing too..

Wondering what's the best way to take the message_field and extract conditional information out of it

2

It's hard to answer without knowing what the input messages look like.

Always format regular expressions (and other configuration) as preformatted text (there's a toolbar button for it). Your current expression has been mangled and isn't useful to us.

3

For my case input can appear in one of the two formats listed below. ( First case tags are present while for other case no tags are present and everything is considered part of message)

Format 1 :
`2017-03-30 09:37:13,705;INFO ;machinename.6028.1.5f80c274-38ad-469f-857f-74e55e1b70fc.1;2.1.1;0;;Tag1.Tag2.Tag3:31; OutArguments are: TransactionSuccess=True;

and other times
Format 2
2017-03-30 09:37:13,743;INFO ;APANDE2-DESK8.6028.1.5f80c274-38ad-469f-857f-74e55e1b70fc.1;2.1.1;0;;calling logOutput for op Message data

I have created a input message filter like following

match=> { "full_message" => "(?[^;])[;](?[^;])[;]%{NUMBER:payloadSize};[;]%{GREEDYDATA:message_text}" }

What i have been trying to achieve is retrieve tag1, tag2 and tag3 when they are present and ignore otherwise.
my approach was to use message_text field created by match statement and then later use this field to retrieve tags (tag1, tag2, tag3) , i know there can be more than one way to achieve i am trying ,so other recommendations are welcome too.

4

So log lines are CSV with either eight or nine column (if I counted them correctly). You could solve the problem with a csv filter and a conditional that checks if the current line had eight or nine fields, but if you want to stick to grok you could use e.g. ((?<tags>[^;]*);)? to optionally match zero or more non-semicolon characters followed by a semicolon.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.