# Filebeat - multiline: Ingest XML's without line feed at end of file

**URL:** <https://discuss.elastic.co/t/filebeat-multiline-ingest-xmls-without-line-feed-at-end-of-file/99021>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [August 31, 2017, 3:17pm UTC](https://discuss.elastic.co/t/filebeat-multiline-ingest-xmls-without-line-feed-at-end-of-file/99021 "2017-08-31T15:17:33Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![cvanhalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cvanhalt/32/21745_2.png) [@cvanhalt](https://discuss.elastic.co/u/cvanhalt)\
**Post date:** [August 31, 2017, 3:17pm UTC](https://discuss.elastic.co/t/filebeat-multiline-ingest-xmls-without-line-feed-at-end-of-file/99021/1 "2017-08-31T15:17:33Z")

</div>

I want to ingest XML files to the ELK-Stack. I want one event per XML file. These XML files end without line feed, thus filebeat's multiline codec never forwards the last line of the XML to Logstash. Because of this Logstash's XML filter is then not able to parse the XML correctly.

I'm using filebeat 5.2.1.

My XML's look like this (I inserted line feeds (LF) to show):

```
LF
<taskReport>LF
LF
  <toplevelinfo Error="0" Warning="0"/>LF
LF
</taskReport>

```

My filebeat.yml looks like this:

```
filebeat.prospectors:
- 
      paths:
        - C:\*.xml
      input_type: log
      document_type: xml
      multiline:
        pattern: '^<taskReport>'
        negate: true
        match: after

output.logstash:
  hosts: ["LS:5044"]

```

Is there an option in filebeat to send this last line within the event?

If I manually add a line feed at the end of the XML logstash can perfectly parse the XML, but this is not an option to me.

Thanks in advance,  
Chris

---

<div class="post-metadata">

**Author:** ![immavalls](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/immavalls/32/146501_2.png) [@immavalls](https://discuss.elastic.co/u/immavalls)\
**Post date:** [August 31, 2017, 4:20pm UTC](https://discuss.elastic.co/t/filebeat-multiline-ingest-xmls-without-line-feed-at-end-of-file/99021/2 "2017-08-31T16:20:44Z")

</div>

I think in your case you can try close\_eof. I understand each XML event is in a file?

[https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html)

---

<div class="post-metadata">

**Author:** ![cvanhalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cvanhalt/32/21745_2.png) [@cvanhalt](https://discuss.elastic.co/u/cvanhalt)\
**Post date:** [September 1, 2017, 7:49am UTC](https://discuss.elastic.co/t/filebeat-multiline-ingest-xmls-without-line-feed-at-end-of-file/99021/3 "2017-09-01T07:49:00Z")

</div>

Yes, each file should be handled as one event.

close\_eof option seems to have no effect on my problem.  
Logstash error is still: **\<REXML::ParseException: No close tag for /taskReport\>**

New filebeat.yml:

```
filebeat.prospectors:
- 
  paths:
    - C:\*.xml
  input_type: log
  document_type: xml
  multiline:
    pattern: '^<taskReport>'
    negate: true
    match: after
  close_eof: true
```

---

<div class="post-metadata">

**Author:** ![immavalls](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/immavalls/32/146501_2.png) [@immavalls](https://discuss.elastic.co/u/immavalls)\
**Post date:** [September 1, 2017, 10:10am UTC](https://discuss.elastic.co/t/filebeat-multiline-ingest-xmls-without-line-feed-at-end-of-file/99021/4 "2017-09-01T10:10:46Z")

</div>

Try `match: before`

---

<div class="post-metadata">

**Author:** ![cvanhalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cvanhalt/32/21745_2.png) [@cvanhalt](https://discuss.elastic.co/u/cvanhalt)\
**Post date:** [September 1, 2017, 12:41pm UTC](https://discuss.elastic.co/t/filebeat-multiline-ingest-xmls-without-line-feed-at-end-of-file/99021/5 "2017-09-01T12:41:54Z")

</div>

I tried every possible combination, also with:

`pattern: '^</taskReport>'`

Sadly none of them fixes the problem.

---

<div class="post-metadata">

**Author:** ![boyersnet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/boyersnet/32/22041_2.png) [@boyersnet](https://discuss.elastic.co/u/boyersnet)\
**Post date:** [September 12, 2017, 7:42pm UTC](https://discuss.elastic.co/t/filebeat-multiline-ingest-xmls-without-line-feed-at-end-of-file/99021/6 "2017-09-12T19:42:29Z")

</div>

I'm having similar issues with ELMAH error logs. My tag is not sent when I run filebeat with the publish option and I'm seeing missing messages in my logstash log file. When I manually add a crlf to the end of the file, it works as I'm expecting.

> - input\_type: log  
> paths:
> - C:\Logs\RESAP\API\*.xml  
> document\_type: ELMAHLog  
> multiline.pattern: ''  
> multiline.negate: true  
> multiline.match: before  
> ignore\_older: 5m  
> close\_eof: true

---

<div class="post-metadata">

**Author:** ![cvanhalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cvanhalt/32/21745_2.png) [@cvanhalt](https://discuss.elastic.co/u/cvanhalt)\
**Post date:** [September 18, 2017, 3:22pm UTC](https://discuss.elastic.co/t/filebeat-multiline-ingest-xmls-without-line-feed-at-end-of-file/99021/7 "2017-09-18T15:22:59Z")

</div>

Resolution for me was to write a script that puts a line feed at the end of each XML.  
Not pretty, but it works.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 16, 2017, 3:23pm UTC](https://discuss.elastic.co/t/filebeat-multiline-ingest-xmls-without-line-feed-at-end-of-file/99021/8 "2017-10-16T15:23:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.