GROK pattern extract

[Sumit_Sijaria]

HI,

I am extracting info from string -

[Fri Dec 01 09:00:00.983 2017] app-pr.com app-onl-pr-lb app-v1-online-pr-node4 0.017070 200 /rest/internal/itemInfo/123123213/method_call

GROK pattern:

[%{DATA:timestamp}] %{DATA:app_url} %{DATA:app_lb} %{DATA:app_node} %{NUMBER:response_time} %{DATA:status} /rest/%{WORD:mthod_type}/itemInfo/%{DATA:item_number}/%{WORD:item_info_method}

Output-

{
"timestamp": [
[
"Fri Dec 01 09:00:00.983 2017"
]
],
"app_url": [
[
"app-pr.com"
]
],
"app_lb": [
[
"app-onl-pr-lb"
]
],
"app_node": [
[
"app-v1-online-pr-node4"
]
],
"response_time": [
[
"0.017070"
]
],
"BASE10NUM": [
[
"0.017070"
]
],
"status": [
[
"200"
]
],
"mthod_type": [
[
"internal"
]
],
"item_number": [
[
"123123213"
]
],
"item_info_method": [
[
"method_call"
]
]
}

Issue- I am not able to preserve the request URL also. I want additional output with field request url as:

/rest/internal/itemInfo/123123213/method_call

Is there a way to do achieve the same. Thanks in advance

[Christian_Dahlqvist]

Using multiple DATA and/or GREEDYDATA patterns can be slow and give incorrect results. It looks like most of your DATA patterns (except for the timestamp field) could be replaced with the more selecting NOTSPACE pattern. To get the url as well as the component, capture the full url in your current pattern, and then apply another grok or dissect filter just on this url field to extract the components.

[Sumit_Sijaria]

Hi Christian,

Thanks for response. Can you please provide example with reference to my query. I mean how to nest GROK inside another GROK

[Christian_Dahlqvist]

Don't try to nest - use two separate grok filters, one working on the message field and one on the extracted url field.

[Sumit_Sijaria]

Will it produce duplicate results in Kibana? I am worried it will lead to incorrect count of extracted URL.

[Christian_Dahlqvist]

Try something like this (not tested):

grok {
	match => { "message" => "[%{DATA:timestamp}] %{NOTSPACE:app_url} %{NOTSPACE:app_lb} %{NOTSPACE:app_node} %{NUMBER:response_time} %{NOTSPACE:status} %{GREEDYDATA:url}" }
}

grok {
	match => { "url" => "/rest/%{WORD:mthod_type}/itemInfo/%{DATA:item_number}/%{WORD:item_info_method}" }
}

[Sumit_Sijaria]

Thanks a lot I will try and let you know