Grok filter logstash config

Elastic Stack Logstash
#1

Hi,
Can somebody help me out here?
So i am trying to use grok filter on a particular log file and I am having some trouble.

I used http://grokdebug.herokuapp.com/ to make a pattern, but I am having trouble with it

so I have the following log entry
10:47:57,434 INFO [ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)' access:14 - |rio|provisionWSG|https://129.4.7.96:543/mobile20/v1/provision/navo

I am having trouble extracting details from the following:
|rio|provisionWSG|https://129.4.7.96:543/mobile20/v1/provision/navo

I am not able to separate the entire string and turn them into fields

I tried
{
"USERNAME": [
[
"rio"
]
],
"NOTSPACE": [
[
"|provisionWSG|https://129.4.7.96:543/mobile20/v1/provision/navo"
]
]
}

please help!

#2

What grok expression do you have so far?

1 Like
#3

For this line: |rio|provisionWSG|https://129.4.7.96:543/mobile20/v1/provision/navo

I have:
%{USERNAME}%{NOTSPACE}

I have this for the entire line:
10:47:57,434 INFO [ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)' access:14 - |rio|provisionWSG|https://129.4.7.96:543/mobile20/v1/provision/navo

%{TIME},%{NUMBER} %{SPACE}%{WORD} %{NOTSPACE} %{SPACE}%{WORD:}%{NOTSPACE} %{NOTSPACE:Threadnumber} %{WORD} %{WORD}%{NOTSPACE} %{NOTSPACE} %{NOTSPACE} %{NOTSPACE} %{NOTSPACE} %{NOTSPACE}%{SYSLOGHOST}

#4

As I believe I said in another thread, you need to match the "|" characters. For your first example something like

\| %{NOTSPACE} \| %{NOTSPACE} \| %{NOTSPACE}

would work.

1 Like
#5

Thank you so much!
This is what exactly I needed!

#6

Hi,
Writing the queries in the grok filter will be bit of hassle if we have more complicated logs. Is there any alternatives I can use so that the fields are extracted from the logs without manually writing a query in logstash file with grok filter?

Please let me know! Thanks.

#7

Logstash doesn't contain any magic feature that parses your logs. You will, somehow, have to describe how they should be interpreted. Sometimes a grok filter is the best tool, other times a kv or csv filter is better, and sometimes you need a combination of these filters.

#8