# Parse XML by Document not Element

**URL:** https://discuss.elastic.co/t/parse-xml-by-document-not-element/288028
**Category:** Logstash
**Created:** [October 29, 2021, 5:56pm UTC](https://discuss.elastic.co/t/parse-xml-by-document-not-element/288028 "2021-10-29T17:56:05Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Tim\_Mobley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tim_mobley/32/94741_2.png) [@Tim\_Mobley](https://discuss.elastic.co/u/Tim_Mobley)
#### Post date: [October 29, 2021, 5:56pm UTC](https://discuss.elastic.co/t/parse-xml-by-document-not-element/288028/1 "2021-10-29T17:56:05Z")

</div>

As a test, I am trying to ingest the following simple XML file (the actual production file is huge).

```auto
<note>
<to>Tove</to>
<from>Jani</from>
<heading>Reminder</heading>
<body>Don't forget me this weekend!</body>
</note>

```

...using the following Logstash pipeline configuration file:

```auto
input {
  file {
    path => ["C:/temp/TEST/*.xml"]
	start_position => "beginning"
  }
}

filter {
  xml {
    source => "message"
	target => "doc"
  }
}

output {
  elasticsearch {
  hosts => ["localhost:9200"]
  index => "test-results-%{+YYYY.MM.dd}"
  }
}

```

In Kibana, each element of the XML file shows up as its own "event", in this case there were 5 XML elements (e.g., \<heading\>\</heading\>), so there are 5 hits:

 ![xml-parsing-per-line](https://us1.discourse-cdn.com/elastic/original/3X/3/c/3cb05fc095a30aa12c1350e1abe5be37ad27776b.png)

What I want instead is to have one "hit" per **document** , with the XML elements as **fields**. So using the simple file as an example, there would be one hit with 5 fields. Is this possible?

Maybe part of the problem is that I'm not clear on how the [source](https://www.elastic.co/guide/en/logstash/current/plugins-filters-xml.html#plugins-filters-xml-source) and [target](https://www.elastic.co/guide/en/logstash/current/plugins-filters-xml.html#plugins-filters-xml-target) settings are to be used. That is, how do you create fields and then put the XML elements in the fields (for instance with [xpath](https://www.elastic.co/guide/en/logstash/current/plugins-filters-xml.html#plugins-filters-xml-xpath))?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [October 29, 2021, 7:57pm UTC](https://discuss.elastic.co/t/parse-xml-by-document-not-element/288028/2 "2021-10-29T19:57:52Z")

</div>

Use a multiline codec on the file input as described [here](https://discuss.elastic.co/t/pasrse-data-between-repeated-xml-tags/172270/2), so that the XML is in a single event.

---

<div class="post-metadata">

### Author: ![INS](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ins/32/92827_2.png) [@INS](https://discuss.elastic.co/u/INS)
#### Post date: [October 29, 2021, 9:44pm UTC](https://discuss.elastic.co/t/parse-xml-by-document-not-element/288028/3 "2021-10-29T21:44:03Z")

</div>

and also You can find the align answer [here](https://discuss.elastic.co/t/how-to-parase-xml-in-logstash/285214/17)

---

<div class="post-metadata">

### Author: ![Tim\_Mobley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tim_mobley/32/94741_2.png) [@Tim\_Mobley](https://discuss.elastic.co/u/Tim_Mobley)
#### Post date: [October 29, 2021, 10:17pm UTC](https://discuss.elastic.co/t/parse-xml-by-document-not-element/288028/4 "2021-10-29T22:17:28Z")

</div>

@Badger thank you! Two follow-ups if you don't mind:

1. For this test, I'm using file input, but the end state will be beats input (from Filebeat on a file server), and [this doc](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-beats.html#plugins-inputs-beats-multiline) says multi-line codecs don't work with Beats input. Will the [multi-line settings](https://www.elastic.co/guide/en/beats/filebeat/7.15/multiline-examples.html#multiline-examples) in filebeat.yml also work for my XML files?
2. Can I still use the xpath filter plugin to parse XML that has been collapsed to one line?

---

<div class="post-metadata">

### Author: ![INS](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ins/32/92827_2.png) [@INS](https://discuss.elastic.co/u/INS)
#### Post date: [October 29, 2021, 10:19pm UTC](https://discuss.elastic.co/t/parse-xml-by-document-not-element/288028/5 "2021-10-29T22:19:07Z")

</div>

right now I'm looking for the solution for reading mass of xml file to parsing per document it means by file EOF (because it's related to internal xml attribute) when Logstash is reading files it thought that he can process a few file in the same chain. But the output such of operation is not desired.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [October 29, 2021, 10:32pm UTC](https://discuss.elastic.co/t/parse-xml-by-document-not-element/288028/6 "2021-10-29T22:32:21Z")

</div>

1. I am not using filebeat but I believe that it has all the same multiline functionality that the codec does.

2. Absolutely.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 26, 2021, 10:32pm UTC](https://discuss.elastic.co/t/parse-xml-by-document-not-element/288028/7 "2021-11-26T22:32:56Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
