# Handle nested xml file with logstash and ruby script

**URL:** <https://discuss.elastic.co/t/handle-nested-xml-file-with-logstash-and-ruby-script/153603>\
**Category:** Logstash\
**Created:** [October 23, 2018, 1:06pm UTC](https://discuss.elastic.co/t/handle-nested-xml-file-with-logstash-and-ruby-script/153603 "2018-10-23T13:06:30Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![humalog](https://avatars.discourse-cdn.com/v4/letter/h/cc9497/32.png) [@humalog](https://discuss.elastic.co/u/humalog)\
**Post date:** [October 23, 2018, 1:06pm UTC](https://discuss.elastic.co/t/handle-nested-xml-file-with-logstash-and-ruby-script/153603/1 "2018-10-23T13:06:30Z")

</div>

Hello,  
I have an XML structure

```auto
<?xml version="1.0" ?>
<XMLdata>
    <Policy>
        <policyName>Arena Standard</policyName>
            <Preferences>
                <ServerPreferences>max_simult_tcp_sessions></ServerPreferences>
            </Preferences>
    </Policy>
    <Report name="Scan">
        <ReportHost name="huit.com">
            <HostProperties>
                <tag name="LastUnauthenticatedResults">111111</tag>
                <tag name="Credentialed_Scan">false</tag>
            </HostProperties>
            <ReportItem port="0" svc_name="general">
                <description>55555</description>
                <risk>none</risk>
            </ReportItem>
        </ReportHost>
        <ReportHost name="1.2.3.4">
            <HostProperties>
                <tag name="LastUnauthenticatedResults">22222</tag>
                <tag name="Credentialed_Scan">true</tag>
            </HostProperties>
            <ReportItem port="15672" svc_name="general">
                <description>9999</description>
                <risk>none</risk>
            </ReportItem>
        </ReportHost>
    </Report>
</XMLdata>

```

I’d like Logstash to output this structure

```auto
{
 "name": "huits.com",
 "LastUnauthenticatedResults": "111111",
 "Credentialed_Scan": "false",
 "Port": "0",
 "svc_name": "general",
 "description": "55555",
 "risk": "none"
}

```

I tried it with xml and ruby filters like the guy in [Stackoverflow](https://stackoverflow.com/questions/45938870/handle-nested-object-with-logstah-and-xpath) posted.  
This is my cfg:

```auto
input {
  file {
    path => "/home/vagrant/data/test.xml"
    start_position => "beginning"
    sincedb_path => "/dev/null"
    codec => multiline {
      pattern => "<XMLdata>"
      negate => "true"
      what => "previous"
      auto_flush_interval => 1
      max_lines => 333333
    }
  }
}

filter {
  xml {
    store_xml => "false"
    source => "message"
    target => "parsed"
  }
  ruby {
    code => '
     # event.set("Preference", event.get("[parsed][ServerPreferences][0][ServerPreverences]"))
     # event.set("HostProperties", event.get("[parsed][ReportHost][HostProperties][0][tag]"))
       event.set("HostName", event.get("[parsed][Report][0][name]"))
       event.set("Port", event.get("[parsed][Report][ReportHost][ReportItem][0][port]"))
    '
  }
  mutate {
  remove_field => ["parsed","@version","message"]
  }
}

output {
  stdout { }
}

```

This is my output

```auto
{
    "@timestamp" => 2018-10-23T12:57:04.661Z,
          "tags" => [
        [0] "multiline"
    ],
      "HostName" => nil,
          "path" => "/home/vagrant/test.xml",
          "Port" => nil,
          "host" => "localhost"
}

```

Why I just get _nil_ when i expect my values. Am I using ruby wrong? I hope for some help.

---

<div class="post-metadata">

**Author:** ![wwalker](https://avatars.discourse-cdn.com/v4/letter/w/43a26b/32.png) [@wwalker](https://discuss.elastic.co/u/wwalker)\
**Post date:** [October 24, 2018, 8:27pm UTC](https://discuss.elastic.co/t/handle-nested-xml-file-with-logstash-and-ruby-script/153603/2 "2018-10-24T20:27:18Z")

</div>

Get rid of the Ruby code and use the xpath function of the XML filter. Below is between 90-100% of it. I'm not quite sure on the syntax of selecting attributes (IE ReportHost **name="[huit.com](http://huit.com)"**) so you may need to tweek that.

```
filter {
  xml {
    store_xml => "false"
    source => "message"
    xpath => [
      "/XMLData/Report/ReportHost/@name", "Name",
      "/XMLData/Report/ReportHost/HostProperties/tag name="LastUnauthenticatedResults"/text()", "LastUnauthenticatedResults",
      "/XMLData/Report/ReportHost/HostProperties/tag name="Credentialed_Scan"/text()", "Credentialed_Scan",
      "/XMLData/Report/ReportHost/ReportItem/@port", "Port",
      "/XMLData/Report/ReportHost/ReportItem/@svc_name", "Svc_name",
      "/XMLData/Report/ReportHost/ReportItem/description/text()", "Description",
      "/XMLData/Report/ReportHost/ReportItem/risk/text()", "Risk"
    ]
  }
}
```

---

<div class="post-metadata">

**Author:** ![humalog](https://avatars.discourse-cdn.com/v4/letter/h/cc9497/32.png) [@humalog](https://discuss.elastic.co/u/humalog)\
**Post date:** [October 25, 2018, 6:36am UTC](https://discuss.elastic.co/t/handle-nested-xml-file-with-logstash-and-ruby-script/153603/3 "2018-10-25T06:36:43Z")

</div>

I had 70% of the XML filter, but i didn't know how to get the attributes of the element . Thank you.

---

<div class="post-metadata">

**Author:** ![balumurari1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/balumurari1/32/39203_2.png) [@balumurari1](https://discuss.elastic.co/u/balumurari1)\
**Post date:** [October 25, 2018, 6:45am UTC](https://discuss.elastic.co/t/handle-nested-xml-file-with-logstash-and-ruby-script/153603/4 "2018-10-25T06:45:53Z")

</div>

Hello humalog,

You can extend the below code to get the output required,

> [@Is it possible to change output structure](https://discuss.elastic.co/t/is-it-possible-to-change-output-structure/153759/6):
>
> The xml file which you have posted has an error, didnot close the tag \<tag pluginname="LastUnauthenticatedResults"\>1539\</tag\> The input code is as follows as per your requirement, input { file { path =\> "D:/xxxxx/ELKStack/sample.xml" start\_position =\> "beginning" sincedb\_path =\> "NUL" codec =\> multiline { pattern =\> "" negate =\> "true" what =\> "previous" auto\_flush\_interval =\> 1 max\_lines =\> 333333 } } } filter { xml { source =\> "message" target =\> "parsed" store\_xml =\> "fal…

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 22, 2018, 6:46am UTC](https://discuss.elastic.co/t/handle-nested-xml-file-with-logstash-and-ruby-script/153603/5 "2018-11-22T06:46:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.