Grok pattern from within kibana dev tools

Elastic Stack Kibana
1

I am configuring an ingest pipeline from within the Kibana Dev Tools and I am having issues with grok pattern, apparently some characters require to be preceeded by a double backslash such as "\\[", and DATA pattern gives a greedy behavor.

lets us consider this log line:
2017-10-10 19:51:38.725 INFO 5648 --- [main] com.seizeit.api.service.StoreService : [method: addInit] [userId: 1] [storeId: 1]

and this pipeline/grok configuration:
PUT /_ingest/pipeline/applogs-pipeline
{
"processors": [
{
"grok": {
"field": "message",
"patterns": ["%{TIMESTAMP_ISO8601:datetime} %{LOGLEVEL:loglevel} %{NUMBER:pid}\s+---\s+\[\s*%{DATA:thread}\s*\]\s+%{DATA:class}\s*:\s*%{DATA:log_message}\s*(\[method\s*:\s*%{DATA:method}\s*\])?\s*(\[userId\s*:\s*%{NUMBER:userId}\s*\])?\s*(\[location\s*:\s*%{DATA:location}\s*\])?\s*(?:\n%{GREEDYDATA:stack})?\n*$"]
}
}
]
}

the result is that: method = addInit] [userId: 1] [storeId: 1
it is a greedy behavor... it should be: method = addInit

Am I missing something

Thanks!

2

I played around with this a bit and I think DATA is behaving as expected. I think there is some issue with the (?:\n%{GREEDYDATA:stack})?\n*$ part. If I remove it, the rest of the parsing works as expected.

3

Having lots of DATA and GREEDYDATA patterns can be very inefficient as they match a lot. It can also lead to errors. Try to always use as targeted patterns as possible, e.g. NOTSPACE, NUMBER etc.

4

thanks, I removed the last part and it is working well now!

5

thanks for the advise! I am new to grok and your comment is more than welcome :slight_smile:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.