Hello,
We have the following pattern which includes dynamic fields nested inside a tag(tag3 in the example below) that needs to be parsed & grok seems to be the go-to-filter to achieve this. However, given fields inside the specified tag are generated dynamically, I don't know if there is a way to handle this with grok along & looking for an efficient approach for this problem statement.
Can experts throw light on how this can be done using grok alone if possible or suitable alternatives?
Log pattern -
<tag1>value1</tag1><tag2>value2</tag2><tag3>key.1=value1,key.2=value2,key.3=value3.....key.N=valueN</tag3><tag4>value1</tag4>
The other alternative I'm thinking of is to -
- prase the message using grok as is
- replace "." with "_" using mutate filter
- split each field as [key_1=value key_2=value2 key_3=value3 ....] using mutate filter
- Finally extract those fields using KV filter
I'm not sure if this is an efficient approach as we expect millions of such documents to be ingested in ES and hence need advice on how to approach this problem.
ELK Stack Version - 7.7
Thanks in advance