# Logstash to parse xml file

**URL:** https://discuss.elastic.co/t/logstash-to-parse-xml-file/140147
**Category:** Logstash
**Created:** [July 16, 2018, 12:10pm UTC](https://discuss.elastic.co/t/logstash-to-parse-xml-file/140147 "2018-07-16T12:10:32Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![saisimo02](https://avatars.discourse-cdn.com/v4/letter/s/edb3f5/32.png) [@saisimo02](https://discuss.elastic.co/u/saisimo02)
#### Post date: [July 16, 2018, 12:10pm UTC](https://discuss.elastic.co/t/logstash-to-parse-xml-file/140147/1 "2018-07-16T12:10:32Z")

</div>

I am new on ELK, and I need a help to use Kibana. I have 5 fields; one for the name of data, and for each name there is different type, and for each type there is different value for each object. My xml file is like that (just an example)

> ```
> <Name nameID="xxxxx"> <Type p="1">xxxxx</Type> <Type
> p="2">yyyyy</Type> <Obj id="1"> <Value r="1">2.2</Value> <Value
> r="2">3.2</Value> <Obj id="2"> <Value r="1">0.2</Value> <Value
> r="2">76.2</Value></Name>
> 
> ```

So what I need to do is to get the name and value of each Type for different obj.

Using Logstash I get

 ![Capture%20du%202018-07-16%2013-40-34](https://us1.discourse-cdn.com/elastic/original/3X/b/f/bfa81aff0941dbb20d2e06f83a08b34165d49c9e.png)  
What I would like to get is different ligne like that:  
 ![Capture](https://us1.discourse-cdn.com/elastic/original/3X/9/f/9fd44cc5a692ffebcdb60b3e7554d2cf56fc2d10.PNG)

My logstash.conf

> input {  
> file {  
> path =\> "/home/test/data.xml"  
> start\_position =\> beginning  
> sincedb\_path =\> "/dev/null"  
> codec =\> multiline  
> {  
> pattern =\> "Name\>"  
> negate =\> true  
> what =\> "previous"  
> }  
> }  
> }  
> filter  
> {  
> xml {  
> source =\> "message"  
> target =\> "parsed"  
> add\_tag =\> "xml"  
> xpath =\> [  
> "//Name/@nameID","Name",  
> "//Type/@p","TypeID",  
> "//Type/text()","Type",  
> "//Obj/@id","Obj",  
> "//r/text()","value"]

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 16, 2018, 3:18pm UTC](https://discuss.elastic.co/t/logstash-to-parse-xml-file/140147/2 "2018-07-16T15:18:57Z")

</div>

That's not valid XML (the Obj elements are not terminated). Show us an actual example of input XML and the output either from Kibana's JSON tab or from output { stdout { codec =\> rubydebug } }

Note also that there is nothing in the XML to associate the values with the types. If you instead had a structure like

```
<Type>
<Obj> <Value></Value> <Value> </Value> </Obj>
<Obj> <Value></Value> <Value> </Value> </Obj>
</Type>
<Type>
<Obj> <Value></Value> <Value> </Value> </Obj>
<Obj> <Value></Value> <Value> </Value> </Obj>
</Type>

```

Then you could use something like

```
filter { xml { source => "message" store_xml => true target => "theXML" force_array => false } }
split { field => "[theXML][Type]" }
split { field => "[theXML][Type][Obj]" }
split { field => "[theXML][Type][Obj][Value]" }

```

As it is you will probably need a ruby filter to iterate over the arrays that xpath returns and build clones of the event.

---

<div class="post-metadata">

### Author: ![saisimo02](https://avatars.discourse-cdn.com/v4/letter/s/edb3f5/32.png) [@saisimo02](https://discuss.elastic.co/u/saisimo02)
#### Post date: [July 17, 2018, 7:44am UTC](https://discuss.elastic.co/t/logstash-to-parse-xml-file/140147/3 "2018-07-17T07:44:55Z")

</div>

My XML looks like that

```
<Name nameID="xxxx">
  <Type p="1">xxxxxx</Type>
  <Type p="2">xxxxxx</Type>
  <Value obj="1"> 
    <r p="1">5.94</r>
    <r p="2">62.19</r>
  </Value>
  <Value obj="2"> 
    <r p="1">5.94</r>
    <r p="2">62.19</r>
  </Value>
</Name>
<Name nameID="yyyy">
  <Type p="1">yyyyy</Type>
  <Type p="2">yyyyyy</Type>
  <Type p="3">yyyy</Type>
  <Value obj="1"> 
    <r p="1">54.94</r>
    <r p="2">6.19</r>
    <r p="3">0</r>
  </Value>
</Name>
```

---

<div class="post-metadata">

### Author: ![saisimo02](https://avatars.discourse-cdn.com/v4/letter/s/edb3f5/32.png) [@saisimo02](https://discuss.elastic.co/u/saisimo02)
#### Post date: [July 17, 2018, 10:32am UTC](https://discuss.elastic.co/t/logstash-to-parse-xml-file/140147/4 "2018-07-17T10:32:57Z")

</div>

What I woul like to get is something like that:  
"NameID = name1  
Type = Type1  
obj = obj1  
Value = xx  
"  
"NameID = name1  
Type = Type2  
obj = obj1  
Value = xx  
"  
"NameID = name1  
Type = Type3  
obj = obj1  
Value = xx  
"  
...etc  
and then  
"NameID = name1  
Type = Type1  
obj = obj2  
Value = xx  
"  
"NameID = name1  
Type = Type2  
obj = obj2  
Value = xx  
"  
....etc  
and the same thing for another Name ( "yyyyy")  
Thanks for help

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 17, 2018, 2:40pm UTC](https://discuss.elastic.co/t/logstash-to-parse-xml-file/140147/5 "2018-07-17T14:40:27Z")

</div>

OK, so for an XML object such as

```
<Name nameID="xxxx">
  <Type p="1">xxxxxx</Type>
  <Type p="2">xxxxxx</Type>
  <Value obj="1">
    <r p="1">5.94</r>
    <r p="2">62.19</r>
  </Value>
  <Value obj="2">
    <r p="1">5.94</r>
    <r p="2">62.19</r>
  </Value>
</Name>

```

You can use

```
filter { xml { source => "message" store_xml => true target => "theXML" force_array => false } }
split { field => "[theXML][Type]" }
split { field => "[theXML][Value]" }
split { field => "[theXML][Value][r]" }

```

to get a collection of events such as

```
{
    "theXML" => {
    "nameID" => "xxxx",
      "Type" => {
        "content" => "xxxxxx",
              "p" => "2"
    },
     "Value" => {
          "r" => {
            "content" => "62.19",
                  "p" => "2"
        },
        "obj" => "2"
    }
},
   "message" => "<Name nameID=\"xxxx\">\n <Type p=\"1\">xxxxxx</Type>\n <Type p=\"2\">xxxxxx</Type>\n <Value obj=\"1\"> \n <r p=\"1\">5.94</r>\n <r p=\"2\">62.19</r>\n </Value>\n <Value obj=\"2\"> \n <r p=\"1\">5.94</r>\n <r p=\"2\">62.19</r>\n </Value>\n</Name>\n",
[...]
}

```

Then you can use mutate+rename to move the fields around and mutate+remove\_field to clean up the debris.

---

<div class="post-metadata">

### Author: ![saisimo02](https://avatars.discourse-cdn.com/v4/letter/s/edb3f5/32.png) [@saisimo02](https://discuss.elastic.co/u/saisimo02)
#### Post date: [July 18, 2018, 7:45am UTC](https://discuss.elastic.co/t/logstash-to-parse-xml-file/140147/6 "2018-07-18T07:45:12Z")

</div>

Thank you Badger, that's work but I get a duplicate result: for example I just need a value corresponding to p='1' for the Type with p='1' and not other value.  
With your example I get for example

```
{
    "theXML" => {
    "nameID" => "xxxx",
      "Type" => {
        "content" => "xxxxxx",
              "p" => "1"
    },
     "Value" => {
          "r" => {
            "content" => "62.19",
                  "p" => "2"
        },
        "obj" => "2"
    }
},

```

And I don't need this output. Thanks again

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 15, 2018, 7:56am UTC](https://discuss.elastic.co/t/logstash-to-parse-xml-file/140147/7 "2018-08-15T07:56:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.