# Logstash XML

**URL:** https://discuss.elastic.co/t/logstash-xml/192452
**Category:** Logstash
**Created:** [July 26, 2019, 1:51pm UTC](https://discuss.elastic.co/t/logstash-xml/192452 "2019-07-26T13:51:48Z")
**Posts on this page:** 15
**Page:** 1

<div class="post-metadata">

### Author: ![Imad\_Bouchakour](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/imad_bouchakour/32/57201_2.png) [@Imad\_Bouchakour](https://discuss.elastic.co/u/Imad_Bouchakour)
#### Post date: [July 26, 2019, 1:51pm UTC](https://discuss.elastic.co/t/logstash-xml/192452/1 "2019-07-26T13:51:49Z")

</div>

Hi

i'am trying to parse a xml file with logstash and i having some issues

this is my XML exemple :

```
<books>
 <book>
		<name>HarryPotter</name>
		<id>785</id>
		<numero>7</numero>
	</book>
<book>
		<name>game of thrones</name>
		<id>441</id>
		<numero>3</numero>
	</book>
</books>

```

my .conf

```
     input{
               ..................
               ......
                codec => multiline {
    	      pattern => '^<book' 	
    	      negate => true
    	      what => "previous"
    	    }   
    	}
    }

      xpath =>
                [
                "/book/id/text()", "id",
                "/book/name/integer", "name",
                "/book/numero/integer", "numero",
                ]
        }

```

the first issue is i'am getting some erreur  
exception=\>#\<REXML::ParseException: Missing end tag for '' (got "books")

the second one is that i can't change the type i'am getting string value in elastic for all how to change that

tks

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 26, 2019, 2:01pm UTC](https://discuss.elastic.co/t/logstash-xml/192452/2 "2019-07-26T14:01:30Z")

</div>

You claim to be matching a pattern that never matches. That will not flush an event unless you enable auto\_flush\_interval. Since you are getting an event I infer that you are not using the configuration you claim to be using.

Once you get that part working you will need to update the xpath expressions

```
"/books/book/id/text()", "id",

```

for example.

---

<div class="post-metadata">

### Author: ![Imad\_Bouchakour](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/imad_bouchakour/32/57201_2.png) [@Imad\_Bouchakour](https://discuss.elastic.co/u/Imad_Bouchakour)
#### Post date: [July 26, 2019, 2:04pm UTC](https://discuss.elastic.co/t/logstash-xml/192452/3 "2019-07-26T14:04:40Z")

</div>

i just fixed my post

```auto
codec => multiline {
    	      pattern => '^<book' 	
    	      negate => true
    	      what => "previous"
    	    } 

```

So ?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 26, 2019, 2:56pm UTC](https://discuss.elastic.co/t/logstash-xml/192452/4 "2019-07-26T14:56:57Z")

</div>

That will result in an event that has this message field

```
<books>\n <book>\n <name>HarryPotter</name>\n <id>785</id>\n <numero>7</numero>\n </book>

```

That is not valid XML. You could use [mutate+gsub](https://discuss.elastic.co/t/how-to-handle-xml-file-last-unconsumed-characters-error/188385/11) to fix it.

Since you did not enable auto\_flush\_interval the second book element will never get flushed.

---

<div class="post-metadata">

### Author: ![Imad\_Bouchakour](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/imad_bouchakour/32/57201_2.png) [@Imad\_Bouchakour](https://discuss.elastic.co/u/Imad_Bouchakour)
#### Post date: [July 29, 2019, 8:53am UTC](https://discuss.elastic.co/t/logstash-xml/192452/5 "2019-07-29T08:53:22Z")

</div>

i did enable the auto\_flush\_interval and still the same problem this is what i'am getting in my log :

> Error parsing xml with XmlSimple {:source=\>"message", :value=\>"\n\t\tHarryPotter\n\t\t785\n\t\t7\n\t\t\n", :exception=\>#\<REXML::ParseException: Missing end tag for '' (got "books")

i am using a http poller. how can i fix it with mutate+gsub and Remove line-breaks ?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 29, 2019, 1:24pm UTC](https://discuss.elastic.co/t/logstash-xml/192452/6 "2019-07-29T13:24:14Z")

</div>

You do not need to remove the line breaks. If you want to the post I linked to shows how to do it. You can remove the books tag using

```
mutate { gsub => ["message", "<books>", ""] }
```

---

<div class="post-metadata">

### Author: ![Imad\_Bouchakour](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/imad_bouchakour/32/57201_2.png) [@Imad\_Bouchakour](https://discuss.elastic.co/u/Imad_Bouchakour)
#### Post date: [July 30, 2019, 12:56pm UTC](https://discuss.elastic.co/t/logstash-xml/192452/7 "2019-07-30T12:56:48Z")

</div>

thanks i fix it.

---

<div class="post-metadata">

### Author: ![Imad\_Bouchakour](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/imad_bouchakour/32/57201_2.png) [@Imad\_Bouchakour](https://discuss.elastic.co/u/Imad_Bouchakour)
#### Post date: [July 30, 2019, 3:16pm UTC](https://discuss.elastic.co/t/logstash-xml/192452/8 "2019-07-30T15:16:20Z")

</div>

Another question: i'am ussing logstash 7.2.0 in my test environment and it work bu in another one there is logstash 6.0.1 and that doesn't work it, is it normal ?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 30, 2019, 3:41pm UTC](https://discuss.elastic.co/t/logstash-xml/192452/9 "2019-07-30T15:41:50Z")

</div>

I would expect the same configuration to work. What does not work?

---

<div class="post-metadata">

### Author: ![Imad\_Bouchakour](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/imad_bouchakour/32/57201_2.png) [@Imad\_Bouchakour](https://discuss.elastic.co/u/Imad_Bouchakour)
#### Post date: [July 31, 2019, 1:52pm UTC](https://discuss.elastic.co/t/logstash-xml/192452/10 "2019-07-31T13:52:57Z")

</div>

log stash bloc completely, and i don't have any log and reset log config don't work. it not problem i'am sending the data to elasticsearch from another environment.

i have this :

status online 282673s

i want to split the "online 282673s" i tried split =\> { "status" =\> " " } but that doesn't work how to slip with space ??

---

<div class="post-metadata">

### Author: ![Imad\_Bouchakour](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/imad_bouchakour/32/57201_2.png) [@Imad\_Bouchakour](https://discuss.elastic.co/u/Imad_Bouchakour)
#### Post date: [August 1, 2019, 1:40pm UTC](https://discuss.elastic.co/t/logstash-xml/192452/11 "2019-08-01T13:40:14Z")

</div>

this is what i did and that doesn't work

```
   gsub => ["status", " ", ":"]
    split => { "status" => ":" }
    add_field => {
             "firstPart" => "%{[status][0]}"
             "secondPart" => "%{[status][1]}"
              }

```

this is what i'am getting

secondPart: %{[status][1]}  
firstPart: online:5039s

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 1, 2019, 2:01pm UTC](https://discuss.elastic.co/t/logstash-xml/192452/12 "2019-08-01T14:01:45Z")

</div>

I cannot explain why that would not work. With this configuration

```
filter {
    mutate { add_field => { "status" => "online 282673s" } }
    mutate {
        split => { "status" => " " }
        add_field => {
            "firstPart" => "%{[status][0]}"
            "secondPart" => "%{[status][1]}"
        }
    }
}

```

or this configuration

```
input { generator { count => 1 lines => [''] } }
filter {
    mutate { add_field => { "status" => "online 282673s" } }
    mutate {
        gsub => ["status", " ", ":"]
        split => { "status" => ":" }
        add_field => {
            "firstPart" => "%{[status][0]}"
            "secondPart" => "%{[status][1]}"
        }
    }
}

```

I get

```
 "firstPart" => "online",
"secondPart" => "282673s",
    "status" => [
    [0] "online",
    [1] "282673s"
],
```

---

<div class="post-metadata">

### Author: ![Imad\_Bouchakour](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/imad_bouchakour/32/57201_2.png) [@Imad\_Bouchakour](https://discuss.elastic.co/u/Imad_Bouchakour)
#### Post date: [August 1, 2019, 2:51pm UTC](https://discuss.elastic.co/t/logstash-xml/192452/13 "2019-08-01T14:51:43Z")

</div>

That didn't work with my field, don't now why. so this is what i did and it work

```
mutate { add_field => { "statusbis" => "%{[status]}" } } 
   mutate {
        split => { "statusbis" => " " }
        add_field => {
            "firstPart" => "%{[statusbis][0]}"
            "secondPart" => "%{[statusbis][1]}"
        }
    }

```

And it works tanks for your help

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 1, 2019, 3:44pm UTC](https://discuss.elastic.co/t/logstash-xml/192452/14 "2019-08-01T15:44:13Z")

</div>

All of that would be consistent with the original status field being an array with a single member.

```
    "status" => [
    [0] "online:282673s"
],

```

That would explain why the original mutates did nothing and why it started working when you copied status to another field (which is not an array).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 29, 2019, 3:44pm UTC](https://discuss.elastic.co/t/logstash-xml/192452/15 "2019-08-29T15:44:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
