# Having troubles with XML Filtering

**URL:** <https://discuss.elastic.co/t/having-troubles-with-xml-filtering/173450>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [March 22, 2019, 8:06am UTC](https://discuss.elastic.co/t/having-troubles-with-xml-filtering/173450 "2019-03-22T08:06:10Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![julsss](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/julsss/32/42044_2.png) [@julsss](https://discuss.elastic.co/u/julsss)\
**Post date:** [March 22, 2019, 8:06am UTC](https://discuss.elastic.co/t/having-troubles-with-xml-filtering/173450/1 "2019-03-22T08:06:10Z")

</div>

Good day everyone,

I am new to this technology and I am trying to filter an xml file with the following elements.

 ![XML](https://us1.discourse-cdn.com/elastic/original/3X/6/9/69e5581354148590cf4f51682bf84a98c357857d.png)

Now, when I checked kibana. The first event that I parsed always include the parent tag

 ![kibana](https://us1.discourse-cdn.com/elastic/original/3X/1/7/174f4b0fede7b1565b4636477db4197ac1b964a2.png)

Due to this when I checked my logstash service it always encounter an error

> [2019-03-22T15:47:14,641][WARN][logstash.filters.xml] Error parsing xml with XmlSimple {:source=\>"message", :value=\>"\n \<Event\n xmlns="[http://logging.apache.org/log4j/2.0/events\](http://logging.apache.org/log4j/2.0/events%5C)" thread="scheduling-1" level="DEBUG" loggerName="com.ck.logdemo.AdventureTime" endOfBatch="false" loggerFqcn="org.apache.logging.log4j.spi.AbstractLogger" threadId="33" threadPriority="5"\>\n \<Instant epochSecond="1552441415" nanoOfSecond="434000000"\>\n You rest at an inn.\n ", :exception=\>#\<REXML::ParseException: No close tag for /Events

Can someone please enlighten why I am encountering this scenario.

Here is my config for the filebeat.

document\_type : xml  
multiline.pattern: '^\<Event\n'  
multiline.negate: true  
multiline.match: after  
multiline.flush\_pattern: '\</Event\>'

Thank you in advance.

---

<div class="post-metadata">

**Author:** ![adrisr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adrisr/32/25423_2.png) [@adrisr](https://discuss.elastic.co/u/adrisr)\
**Post date:** [March 22, 2019, 9:38am UTC](https://discuss.elastic.co/t/having-troubles-with-xml-filtering/173450/2 "2019-03-22T09:38:55Z")

</div>

Hi,

There is an error in your `multiline.pattern`, it is currently not matching any lines. You should get rid of the caret `^`.

I tested using this settings:

```auto
  multiline.pattern: '<Event>'
  multiline.negate: true
  multiline.match: after
  multiline.flush_pattern: '</Event>'
  exclude_lines: 'Events>'

```

and this sample file:

```auto
<Events>
  <Event>
    <Message>Line 1</Message>
  </Event>
  <Event>
    <Message>Line 2</Message>
  </Event>
  <Event>
    <Message>Line 3</Message>
  </Event>
</Events>

```

This results in 3 documents being published:

```auto
"message": " <Event>\n <Message>Line 1</Message>\n </Event>",

```

```auto
"message": " <Event>\n <Message>Line 2</Message>\n </Event>",

```

```auto
"message": " <Event>\n <Message>Line 3</Message>\n </Event>",

```

---

<div class="post-metadata">

**Author:** ![julsss](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/julsss/32/42044_2.png) [@julsss](https://discuss.elastic.co/u/julsss)\
**Post date:** [March 22, 2019, 10:09am UTC](https://discuss.elastic.co/t/having-troubles-with-xml-filtering/173450/3 "2019-03-22T10:09:33Z")

</div>

Helllo,

Thank you for the response. I tried the config you have provided. It removes the \<Events\> tag. But apparently the first \<Event\> tag is not included in the document published. This is the whole xml document I am using.

```
> <Events>
> <Event
> xmlns="http://logging.apache.org/log4j/2.0/events" thread="scheduling-1" level="DEBUG" loggerName="com.ck.logdemo.AdventureTime" endOfBatch="false" loggerFqcn="org.apache.logging.log4j.spi.AbstractLogger" threadId="33" threadPriority="5">
> <Instant epochSecond="1552441415" nanoOfSecond="434000000"></Instant>
> <Message>You rest at an inn.</Message>
> </Event>
> <Event
> xmlns="http://logging.apache.org/log4j/2.0/events" thread="scheduling-1" level="ERROR" loggerName="com.ck.logdemo.AdventureTime" endOfBatch="false" loggerFqcn="org.apache.logging.log4j.spi.AbstractLogger" threadId="33" threadPriority="5">
> <Instant epochSecond="1552441417" nanoOfSecond="306000000"></Instant>
> <Message>You levelled up! You are now level 1</Message>
> </Event>
> <Event
> xmlns="http://logging.apache.org/log4j/2.0/events" thread="scheduling-1" level="ERROR" loggerName="com.ck.logdemo.AdventureTime" endOfBatch="false" loggerFqcn="org.apache.logging.log4j.spi.AbstractLogger" threadId="33" threadPriority="5">
> <Instant epochSecond="1552441417" nanoOfSecond="306000000"></Instant>
> <Message>You levelled up! You are now level 2</Message>
> </Event>
> <Event
> xmlns="http://logging.apache.org/log4j/2.0/events" thread="scheduling-1" level="ERROR" loggerName="com.ck.logdemo.AdventureTime" endOfBatch="false" loggerFqcn="org.apache.logging.log4j.spi.AbstractLogger" threadId="33" threadPriority="5">
> <Instant epochSecond="1552441417" nanoOfSecond="306000000"></Instant>
> <Message>You levelled up! You are now level 3</Message>
> </Event>
> <Event
> xmlns="http://logging.apache.org/log4j/2.0/events" thread="scheduling-1" level="ERROR" loggerName="com.ck.logdemo.AdventureTime" endOfBatch="false" loggerFqcn="org.apache.logging.log4j.spi.AbstractLogger" threadId="33" threadPriority="5">
> <Instant epochSecond="1552441417" nanoOfSecond="306000000"></Instant>
> <Message>You levelled up! You are now level 4</Message>
> </Event>
> <Event
> xmlns="http://logging.apache.org/log4j/2.0/events" thread="scheduling-1" level="ERROR" loggerName="com.ck.logdemo.AdventureTime" endOfBatch="false" loggerFqcn="org.apache.logging.log4j.spi.AbstractLogger" threadId="33" threadPriority="5">
> <Instant epochSecond="1552441417" nanoOfSecond="306000000"></Instant>
> <Message>You levelled up! You are now level 5</Message>
> </Event>
> <Event
> xmlns="http://logging.apache.org/log4j/2.0/events" thread="scheduling-1" level="ERROR" loggerName="com.ck.logdemo.AdventureTime" endOfBatch="false" loggerFqcn="org.apache.logging.log4j.spi.AbstractLogger" threadId="33" threadPriority="5">
> <Instant epochSecond="1552441417" nanoOfSecond="306000000"></Instant>
> <Message>You died</Message>
> </Event>
> <Event
> xmlns="http://logging.apache.org/log4j/2.0/events" thread="scheduling-1" level="ERROR" loggerName="com.ck.logdemo.AdventureTime" endOfBatch="false" loggerFqcn="org.apache.logging.log4j.spi.AbstractLogger" threadId="33" threadPriority="5">
> <Instant epochSecond="1552441417" nanoOfSecond="306000000"></Instant>
> <Message>You respawn</Message>
> </Event>
> </Events>

```

The document results is only 7.

The event below was not included in the document result:

```
> <Event
> xmlns="http://logging.apache.org/log4j/2.0/events" thread="scheduling-1" level="DEBUG" loggerName="com.ck.logdemo.AdventureTime" endOfBatch="false" loggerFqcn="org.apache.logging.log4j.spi.AbstractLogger" threadId="33" threadPriority="5">
> <Instant epochSecond="1552441415" nanoOfSecond="434000000"></Instant>
> <Message>You rest at an inn.</Message>
> </Event>
```

---

<div class="post-metadata">

**Author:** ![adrisr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adrisr/32/25423_2.png) [@adrisr](https://discuss.elastic.co/u/adrisr)\
**Post date:** [March 22, 2019, 10:45am UTC](https://discuss.elastic.co/t/having-troubles-with-xml-filtering/173450/4 "2019-03-22T10:45:27Z")

</div>

Oh, I didn't realise that your "" is not found in a single line, so the provided `multiline.pattern` never matches.

Replace with:

```auto
   multiline.pattern: '<Event'

```

---

<div class="post-metadata">

**Author:** ![julsss](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/julsss/32/42044_2.png) [@julsss](https://discuss.elastic.co/u/julsss)\
**Post date:** [March 25, 2019, 2:21am UTC](https://discuss.elastic.co/t/having-troubles-with-xml-filtering/173450/5 "2019-03-25T02:21:55Z")

</div>

Thanks for the response. That solved my problem.

May I know why the \<Events\> tag was included in the first document? Even though I set the multiline.match to after.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 22, 2019, 2:21am UTC](https://discuss.elastic.co/t/having-troubles-with-xml-filtering/173450/6 "2019-04-22T02:21:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.