Grok parser and nested brackets

Elastic Stack Logstash
1

Hi,

I have log event like this

2023-03-03T11:11:11.000Z INFO (foo (bar) bla bla [bla]) 2023-03-03T11:11:11.000Z [foo (bar) bla bla [bla]]

I want to parse it with grok filter like

timestamp: 2023-03-03T11:11:11.000Z
level: INFO
thread: foo (bar) bla bla [bla]
message: 2023-03-03T11:11:11.000Z [foo (bar) bla bla [bla]]

When I use grok parser

%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{LOGLEVEL:level}%{SPACE}\(%{DATA:thread}\)%{SPACE}%{GREEDYDATA:message}

I get

thread: foo (bar

When I use

%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{LOGLEVEL:level}%{SPACE}\(%{GREEDYDATA:thread}\)%{SPACE}%{GREEDYDATA:message}

I get

thread: foo (bar) bla bla [bla]) 2023-03-03T11:11:11.000Z [foo (bar

How should I parse nested brackets to get what I want?

2

I wasn't able to get it 100% what you want but close. If you are able to combine the two message fields in a subsequent step that might get you all the way.

%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:level} %{GREEDYDATA:thread} %{TIMESTAMP_ISO8601:message0} %{GREEDYDATA:message1}
3

Is this a single line or multiline? Is the level field optional?

4

@Rios this a single line

level field is mandatory

but the thread field in log event sometime could be (foo bar bla bla bla)

5

@Wave thank you for your advice, but the second timestamp is not always in message.

The common form of the log record is

2023-03-03T11:11:11.000Z INFO (foo bar bla bla bla) a message text

And I'd like to make the parser more universal

6

You can use something like this:

   grok {
       break_on_match => true
       match => {
       "message" => [
               "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:level} \((?<thread>.*)\) %{TIMESTAMP_ISO8601:timestamp2}%{SPACE}%{GREEDYDATA:msg}",
               "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:level} \((?<thread>.*)\) %{GREEDYDATA:msg}"
               
           ]
       }
   }

It will separate the 2nd date field and rest of data.

1 Like
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.