Logstash expecting \n for it to process an XML

Hello Team,

I am trying to parse through an XML file and read only a single tag out of the entire XML using the Grok Patterns.

My grok pattern looks like this. Its able to parse through an XML when its properly indented, as there's a new line after each closing tag. But when the file comes with no spaces in between consecutive tags, this pattern does not work. Could anyone help here?

filter {
#ignore log comments
if [message] =~ "^#" {
drop {}

grok {
patterns_dir => "./patterns"
match => ["message", "%{DATA:extras}<LoadID%{DATA:extra}>%{DATA:ASNNumber}%{GREEDYDATA:behind}"]
date {
match => [ "timestamp", "yyyy-MM-dd HH:mm:ss" ]
locale => "en"

Second filter

filter {
if "_grokparsefailure" in [tags] {
drop { }
} else {
# on success remove the message field to save space
mutate {
remove_field => ["message", "timestamp", "extra", "extras", "behind"]

I suggest you use an xml filter to parse XML.

I am not really sure how to go about about it?
Can you please suggest changes in my existing filters?
I only need to read the value of the tag
<LoadID></LoadID> or <LoadID xmlns=""></LoadID>

I can't help without knowing what the data looks like. What does a stdout { codec => rubydebug } output produce?

This works fine for me.

<tns:ASNAck xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:tns="http://www.xyx.com/YYY/logistics/mxg/xnsds/V1_0">

This doesn't work:

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?><tns:ASNAck xmlns:tns='http://www.xyx.com/YYY/logistics/mxg/xnsds/V1_0' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'><MessageHeader><MessageID>3031999515</MessageID><MessageTimeStamp>2017-09-12T06:37:36Z</MessageTimeStamp><SenderID>XBHSNS</SenderID><ReceiverID>GOLF_DAO</ReceiverID><MessageType>ACKACKACK</MessageType><CorrelationID>2d323537383935353034383933383135</CorrelationID></MessageHeader><Masterbill>G829441</Masterbill><LoadID>Jitesh555</LoadID><Accept>true</Accept><ReasonCode/><ReasonDescription/></tns:ASNAck>

That's not the output from stdout { codec => rubydebug }. For best results please answer my questions. I'd also like to see the complete configuration (specifically your inputs).

input {
  beats {
    port => 5045
    type => 'iis'


This is my input. There is nothing in the ruby debug as the pattern doesn't match

There is nothing in the ruby debug as the pattern doesn't match

Then stop dropping non-matching events so that all events are routed to the stdout output

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.