# Logstash multiline filter not merging xml after new line

**URL:** <https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827>\
**Category:** Logstash\
**Created:** [August 11, 2016, 2:14pm UTC](https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827 "2016-08-11T14:14:51Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![beelastic](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@beelastic](https://discuss.elastic.co/u/beelastic)\
**Post date:** [August 11, 2016, 2:14pm UTC](https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827/1 "2016-08-11T14:14:51Z")

</div>

I want to parse xml files with logstash  
Example File:

> `<?xml version="1.0" encoding="UTF-8"?>`  
> `<ns1:Alert_E01 xmlns:ns1="urn:contoso.com:elasticSearch:alert"><Alerts><Alert><PI_SID>SPI</PI_SID></Alert></Alerts></ns1:Alert_E01>`

There is a newline (\n) after `<?xml version="1.0" encoding="UTF-8"?>`.

To parse the whole xml in one field i have to use the multiline filter:  
`filter{ multiline { pattern => "\s$" negate => false what => "next" } }`

But it doesnt work, i get only the first line in the message field:  
`message:"<?xml version="1.0" encoding="UTF-8"?>"`

_BUT_ when i make a xml file like this:

```
firstline
secondline
thirdline

```

and parse it, I get  
`message:"firstline secondline thirdline`"  
as expected

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 12, 2016, 6:03am UTC](https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827/2 "2016-08-12T06:03:34Z")

</div>

I'm not following the logic here. You want to join with the next line if the current line ends with a newline character?

---

<div class="post-metadata">

**Author:** ![beelastic](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@beelastic](https://discuss.elastic.co/u/beelastic)\
**Post date:** [August 12, 2016, 6:26am UTC](https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827/3 "2016-08-12T06:26:31Z")

</div>

Yes, if a line ends with newline, i want to join with the next line.  
Because the XML files have newline characters, and to parse the XML i need the whole XML to be in one field.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 12, 2016, 6:31am UTC](https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827/4 "2016-08-12T06:31:10Z")

</div>

Well, with the possible exception of the last line of the file all lines end with a newline character. When using `\s` like you do here I'm not sure it matches the line's newline character. I'd use `^` instead. All lines have a beginning.

---

<div class="post-metadata">

**Author:** ![beelastic](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@beelastic](https://discuss.elastic.co/u/beelastic)\
**Post date:** [August 12, 2016, 6:46am UTC](https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827/5 "2016-08-12T06:46:17Z")

</div>

i tried it with

```
multiline { pattern=>"^.*"
            negate=>false
            what=> "next"
}

```

But get the file splitted after newline.

---

<div class="post-metadata">

**Author:** ![beelastic](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@beelastic](https://discuss.elastic.co/u/beelastic)\
**Post date:** [August 12, 2016, 7:10am UTC](https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827/6 "2016-08-12T07:10:11Z")

</div>

i tried it also with

```
multiline { pattern=>"^<.*"
            negate=>false
            what=> "next"
}

```

and

```
multiline { pattern=>"^<.*"
            negate=>false
            what=> "previous"
}

```

and

```
multiline { pattern=>"^>.*"
            negate=>true
            what=> "next"
}

```

No line have "\>" at the beginning, so if i am following the logic right, every line must be joined with the next line.  
But still get the file splitted after newline:  
`message:"<?xml version="1.0" encoding="UTF-8"?>"`

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 12, 2016, 7:10am UTC](https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827/7 "2016-08-12T07:10:23Z")

</div>

If you provide a complete and reproducible configuration example it'll be easier to help.

---

<div class="post-metadata">

**Author:** ![beelastic](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@beelastic](https://discuss.elastic.co/u/beelastic)\
**Post date:** [August 12, 2016, 7:11am UTC](https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827/8 "2016-08-12T07:11:53Z")

</div>

```
input{
	file{
		path => "C:/XMLdata/*.xml"
		start_position => "beginning"
		sincedb_path => "C:/parsedfiles.sincedb"

	}
}
filter{

   multiline {
      pattern => "^>.*"
      negate => true
      what => "next"
    }
  }

output{
	elasticsearch{
		
		index => "xml-index"
		hosts => "127.0.0.1:9200"
	}
}

```

here it is! 🙂

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [August 12, 2016, 7:19am UTC](https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827/9 "2016-08-12T07:19:38Z")

</div>

I'm not able to reproduce what you describe. The problem I have is actually getting Logstash to emit anything, because if you always join with the next line Logstash won't know when to stop waiting for the next line. I don't have any more time to spend on this. Good luck.

---

<div class="post-metadata">

**Author:** ![beelastic](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@beelastic](https://discuss.elastic.co/u/beelastic)\
**Post date:** [August 12, 2016, 7:20am UTC](https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827/10 "2016-08-12T07:20:17Z")

</div>

Okay, thank you very much for your time 🙂

---

<div class="post-metadata">

**Author:** ![JIELOGAN](https://avatars.discourse-cdn.com/v4/letter/j/67e7ee/32.png) [@JIELOGAN](https://discuss.elastic.co/u/JIELOGAN)\
**Post date:** [May 31, 2017, 7:08am UTC](https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827/11 "2017-05-31T07:08:47Z")

</div>

hello，Are you solved this？

---

<div class="post-metadata">

**Author:** ![beelastic](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@beelastic](https://discuss.elastic.co/u/beelastic)\
**Post date:** [May 31, 2017, 7:23am UTC](https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827/12 "2017-05-31T07:23:21Z")

</div>

No Unfortunately not. I decided to make it without logstash and stored the  
data directly via the elasticsearch API

---

<div class="post-metadata">

**Author:** ![JIELOGAN](https://avatars.discourse-cdn.com/v4/letter/j/67e7ee/32.png) [@JIELOGAN](https://discuss.elastic.co/u/JIELOGAN)\
**Post date:** [May 31, 2017, 7:34am UTC](https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827/13 "2017-05-31T07:34:24Z")

</div>

How to achieve this? could you please show me ?

---

<div class="post-metadata">

**Author:** ![beelastic](https://avatars.discourse-cdn.com/v4/letter/b/bc8723/32.png) [@beelastic](https://discuss.elastic.co/u/beelastic)\
**Post date:** [May 31, 2017, 7:51am UTC](https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827/14 "2017-05-31T07:51:53Z")

</div>

I used SAP as Interface, If you have SAP in your Company your SAP  
colleagues will help you with your data connection 🙂

---

<div class="post-metadata">

**Author:** ![JIELOGAN](https://avatars.discourse-cdn.com/v4/letter/j/67e7ee/32.png) [@JIELOGAN](https://discuss.elastic.co/u/JIELOGAN)\
**Post date:** [May 31, 2017, 7:54am UTC](https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827/15 "2017-05-31T07:54:41Z")

</div>

> [@beelastic](#):
>
> SAP

ok thanks:blush:

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:26am UTC](https://discuss.elastic.co/t/logstash-multiline-filter-not-merging-xml-after-new-line/57827/16 "2017-07-06T04:26:12Z")

</div>