I am trying to extract a log-pattern
[LEVEL] (COMPONENT) MESSAGE [msgid: MESSAGE ID]
example message:
[INFO ] (core.workflow) Start scan, workflow_id='lms::workflow::WorkflowExecutor(0x259c99f94b0, name = "WF-127.0.0.1:60708")', dataId='4d66c5069365476a8f36431e6369ab0b', fileName='winlogbeat.yml' [msgid: 3491]
In the pipeline I have two filters:
(1) Grok: That extracts level, component, kvmessage and mid - This works.
(2) kv: I want to now split values like workflow_id
, name
etc as key value pairs. However, this is not working. Would appreciate some assistance.
[
{
"grok": {
"field": "message",
"patterns": [
"\\[%{WORD:level}\\s*\\]: \\(%{COMPONENT:component}\\) %{MSG:kvmessage}\\[%{GREEDYDATA:mid}\\]"
],
"pattern_definitions": {
"COMPONENT": "\\w*.\\w*",
"STATUS": "\\w*\\s\\w*",
"MSG": "[^\\[]*"
},
"ignore_missing": true
}
},
{
"kv": {
"field": "kvmessage",
"field_split": ",",
"value_split": "="
}
}
]