Parse XML sub tags as a separate log

Elastic Stack Logstash
1

Hi Team,
I have a XML formatted as below

<?xml version="1.0" encoding="UTF-8"?> 
<documents>
<Document><docID>101074476</docID><Title>End of Sale 1403 and 1416</Title><Author>clark13</Author></Document>
<Document><docID>101074474</docID><Title>End of Sale 1406 and 1417</Title><Author>clark14</Author></Document>
</document>

I need each <Document> as a separate log and then use xml filter on it.
I have used multiline codec as below

codec => multiline {
                pattern => "<Document>"
                negate => "true"
                what => "previous"
                }

but its not considering last entry from xml file. To add all logs as event I am adding empty <Document></Document> at last.

I think there is some change needs in multiline codec. Not sure what I can add to consider all from XML.

Thanks,
Disha

2

You are not getting the last entry because the codec will not flush an event until the pattern matches. There is no line that matches the pattern after the last line that does so (obviously), so the last line is never flushed. You could set auto_flush_interval.

Personally I would consume the entire document using a multiline codec, then parse the XML to get an array of Document elements and use a split filter to separate those into one per event.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.