# How to parse XML?

**URL:** <https://discuss.elastic.co/t/how-to-parse-xml/206238>\
**Category:** Logstash\
**Created:** [November 2, 2019, 1:34pm UTC](https://discuss.elastic.co/t/how-to-parse-xml/206238 "2019-11-02T13:34:40Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![ja123](https://avatars.discourse-cdn.com/v4/letter/j/bc8723/32.png) [@ja123](https://discuss.elastic.co/u/ja123)\
**Post date:** [November 2, 2019, 1:34pm UTC](https://discuss.elastic.co/t/how-to-parse-xml/206238/1 "2019-11-02T13:34:40Z")

</div>

I'm trying to parse the following xml from an api through xml plugin.

```
<Journey fpTime="12:33" fpDate="02.11.19" delay="0" e_delay="0" platform="102" targetLoc="abc" dirnr="8001055" prod="S 60#S" dir="Böblingen" administration="800643" depStation="Stuttgart Hbf &#x0028;tief&#x0029;" is_reachable="0" delayReason=" " approxDelay="0"></Journey>
<Journey fpTime="12:33" fpDate="02.11.19" delay="0" e_delay="0" platform="101" targetLoc="Stuttgart Schwabstr." dirnr="8006698" prod="S 5#S" dir="Stuttgart Schwabstr." administration="800643" depStation="Stuttgart Hbf &#x0028;tief&#x0029;" is_reachable="0" delayReason=" " approxDelay="0"></Journey>

```

My config file looks like this:

> ```
> input {
> http_poller {
> urls => {
>      
> test2 => {
> # Supports all options supported by ruby's Manticore HTTP client
> method => get
> url => "https://reiseauskunft.bahn.de/bin/stboard.exe/dn?rt=1&input=Stuttgart%20Hauptbahnhof&boardType=dep&L=vs_java3&productsFilter=111110111&start=yes&maxJourneys=20"
> headers => {
> Accept => "application/json"
> }
> }
> }
> request_timeout => 20
> # Supports "cron", "every", "at" and "in" schedules by rufus scheduler
> schedule => { cron => "* * * * * UTC"}
> codec => "plain"
> # A hash of request metadata info (timing, response headers, etc.) will be sent here
> metadata_target => "http_poller_metadata"
> }
> }
> filter {
> 
> ## interpret the message payload as XML
> xml {
> source => "message"
> target => "parsed"
> store_xml => true 
> force_array => false 
> }
> 
> split {
> field => "[parsed][Journey]"
> add_field => {
> ## generate a unique id for the station # X the sensor time to prevent duplicates
> id => "%{[parsed][Journey][fpTime]}-%{[parsed][Journey][fpDate]}-%{[parsed][Journey][dirnr]"
> targetStationName => "%{[parsed][Journey][targetLoc]}"
> time => "%{[parsed][Journey][fpTime]}"
> dir => "%{[parsed][Journey][dirnr]}"
> jDate => "%{[parsed][Journey][fpDate]}"
> e_delay => "%{[parsed][Journey][e_delay]}"
> depStation => "%{[parsed][Journey][depStation]}"
> delayReason => "%{[parsed][Journey][delayReason]}"
> administration => "%{[parsed][Journey][administration]}"
> prod	=> "%{[parsed][Journey][prod]}"
> platform	=> "%{[parsed][Journey][platform]}"
> }
> }
> mutate {
> ## Convert the numeric fileds to the appropriate data type from strings
> convert => {
> "e_delay" => "integer"
> }
> ## put the geospatial value in the correct [longitude, latitude] format
> add_field => { "fullDate" => ["%{[date]}", "%{[time]}" ]}
> ## get rid of the extra fields we don't need
> remove_field => ["message", "parsed", "http_poller_metadata"]
> }
> ## use the embedded Unix timestamp 
> date {
> match => ["fullDate", "UNIX_MS"]
> remove_field => ["jDate"]
> }
> }
> 
> ```

I want to parse each element in` <Journey>` seperatley but I get the the error message :exception=\>#\<REXML::ParseException: missing attribute quote  
Line: 1  
Position: 6750  
Last 80 unconsumed characters:

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [November 2, 2019, 2:50pm UTC](https://discuss.elastic.co/t/how-to-parse-xml/206238/2 "2019-11-02T14:50:52Z")

</div>

If you supplied the header 'Accept =\> "application/json"' I would be surprised if the API returned XML.

If it does return the XML you showed then there will not be a [parsed][Journey] field, so your split filter does nothing. "%{[parsed][Journey][delayReason]}" should be "%{[parsed][delayReason]}" etc.

None of that explains the missing attribute quote error.

---

<div class="post-metadata">

**Author:** ![ja123](https://avatars.discourse-cdn.com/v4/letter/j/bc8723/32.png) [@ja123](https://discuss.elastic.co/u/ja123)\
**Post date:** [November 6, 2019, 11:29am UTC](https://discuss.elastic.co/t/how-to-parse-xml/206238/3 "2019-11-06T11:29:46Z")

</div>

I could fix the error now my config looks like

```
input {
  http_poller {
    urls => {
     
      test2 => {
        method => get
         url => "https://reiseauskunft.bahn.de/bin/stboard.exe/dn?rt=1&input=Stuttgart%20Hauptbahnhof&boardType=dep&L=vs_java3&productsFilter=111110111&start=yes&maxJourneys=20"
     
     }
	   
    }

	
	 codec => multiline
        {
            pattern => "<Journey"
            negate => true
            what => "previous"
			charset => "ISO-8859-1"
			auto_flush_interval => 1
        }
	 
    request_timeout => 20
    schedule => { cron => "* * * * * UTC"}
	
    # A hash of request metadata info (timing, response headers, etc.) will be sent here
    metadata_target => "http_poller_metadata"
  }
}
filter {

 xml {
    remove_namespaces => true
    source => "message"
	force_array => false
	target => "msg"
	xpath => ["/Journey/@fpTime", "jTime"]
  }
  
}

```

When I run Logstash i get the following error. Do you know how to fix this?

exception=\>#\<REXML::ParseException: #\<RuntimeError: attempted adding second root element to document\>Line: 1  
Position: 7305

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [November 6, 2019, 2:45pm UTC](https://discuss.elastic.co/t/how-to-parse-xml/206238/4 "2019-11-06T14:45:27Z")

</div>

> [@ja123](#):
>
> exception=\>#\<REXML::ParseException: #\<RuntimeError: attempted adding second root element to document

I believe that is what you get if you try to parse something like

```
<item>foo</item><item>bar</item>

```

An xml filter will only work if the input is something like

```
<items><item>foo</item><item>bar</item></items>

```

---

<div class="post-metadata">

**Author:** ![ja123](https://avatars.discourse-cdn.com/v4/letter/j/bc8723/32.png) [@ja123](https://discuss.elastic.co/u/ja123)\
**Post date:** [November 6, 2019, 5:03pm UTC](https://discuss.elastic.co/t/how-to-parse-xml/206238/5 "2019-11-06T17:03:52Z")

</div>

And which type of filter can I use instead?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [November 6, 2019, 5:24pm UTC](https://discuss.elastic.co/t/how-to-parse-xml/206238/6 "2019-11-06T17:24:20Z")

</div>

Use mutate+gsub to add an element wrapping both of the existing elements.

---

<div class="post-metadata">

**Author:** ![ja123](https://avatars.discourse-cdn.com/v4/letter/j/bc8723/32.png) [@ja123](https://discuss.elastic.co/u/ja123)\
**Post date:** [November 7, 2019, 12:37pm UTC](https://discuss.elastic.co/t/how-to-parse-xml/206238/7 "2019-11-07T12:37:29Z")

</div>

Ok, and how can I add an element at the beginning and at the end position of the message.  
filter {  
mutate { gsub =\> ["message", "Start of Message", ""] }  
}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [November 7, 2019, 2:58pm UTC](https://discuss.elastic.co/t/how-to-parse-xml/206238/8 "2019-11-07T14:58:56Z")

</div>

Use anchors

```
mutate { gsub => ["message", "^", "<a>", "message", "$", "</a>"] }
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 5, 2019, 2:58pm UTC](https://discuss.elastic.co/t/how-to-parse-xml/206238/9 "2019-12-05T14:58:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.