# Parsing nested XML with logstash

**URL:** <https://discuss.elastic.co/t/parsing-nested-xml-with-logstash/202530>\
**Category:** Logstash\
**Created:** [October 7, 2019, 12:37pm UTC](https://discuss.elastic.co/t/parsing-nested-xml-with-logstash/202530 "2019-10-07T12:37:54Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![jsj](https://avatars.discourse-cdn.com/v4/letter/j/7ea924/32.png) [@jsj](https://discuss.elastic.co/u/jsj)\
**Post date:** [October 7, 2019, 12:37pm UTC](https://discuss.elastic.co/t/parsing-nested-xml-with-logstash/202530/1 "2019-10-07T12:37:54Z")

</div>

Hi,

I'm parsing xml-data from endpoint system with logstash and i'm not sure how parse nested data from xml;

Here's the sample data & logstash config

```
 <eventItem sequence_num="206423128" uid="18960533">
  <timestamp>2019-02-04T08:18:55.430Z</timestamp>
  <eventType>processEvent</eventType>
  <details>
   <detail>
    <name>eventType</name>
    <value>end</value>
   </detail>
   <detail>
    <name>pid</name>
    <value>1188</value>
   </detail>
   <detail>
    <name>processPath</name>
    <value>C:\Windows\System32\svchost.exe</value>
   </detail>
   <detail>
    <name>process</name>
    <value>svchost.exe</value>
   </detail>
   <detail>
    <name>parentPid</name>
    <value>792</value>
   </detail>
   <detail>
    <name>parentProcessPath</name>
    <value>C:\Windows\System32\services.exe</value>
   </detail>
   <detail>
    <name>parentProcess</name>
    <value>services.exe</value>
   </detail>
   <detail>
    <name>username</name>
    <value>user1234</value>
   </detail>
   <detail>
    <name>startTime</name>
    <value>2019-02-04T08:18:55.430Z</value>
   </detail>
  </details>
 </eventItem>

```

And the logstash config at the moment:

```
input {
    file {
        path => "/path/to/data/testdataset.xml"
        start_position => "beginning"
    sincedb_path => "/dev/null"
    codec => multiline {
        pattern => "^\s<eventItem" 
        negate => "true"
        what => "previous"
    } 
    }
}

filter {
    xml {
    source => "message"
    store_xml => true
    target => "agentevent"
    #xpath => ["/eventItems/eventType/text()", "eventType"]
    }
}

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "xml_test"
  }
}

```

This produced result that i'm looking, except for the details; This is where the details from event are populated and the number of details varies by the type of event.

At the moment the events parse like following:

```
"agentevent": {
  "details": [
    {
      "detail": [
        {
          "value": [
            "end"
          ],
          "name": [
            "eventType"
          ]
        },
        {
          "value": [
            "1188"
          ],
          "name": [
            "pid"
          ]
        },
        {
          "value": [
            "C:\\Windows\\System32\\svchost.exe"
          ],
          "name": [
            "processPath"
          ]
        },
        {
          "value": [
            "svchost.exe"
          ],
          "name": [
            "process"
          ]
        },
        {
          "value": [
            "792"
          ],
          "name": [
            "parentPid"
          ]
        },
        {
          "value": [
            "C:\\Windows\\System32\\services.exe"
          ],
          "name": [
            "parentProcessPath"
          ]
        },
        {
          "value": [
            "services.exe"
          ],
          "name": [
            "parentProcess"
          ]
        },
        {
          "value": [
            "user1234"
          ],
          "name": [
            "username"
          ]
        },
        {
          "value": [
            "2019-02-04T08:18:55.430Z"
          ],
          "name": [
            "startTime"
          ]
        }
      ]
    }
  ],
  "sequence_num": "206423128",
  "uid": "18960533",
  "timestamp": [
    "2019-02-04T08:18:55.430Z"
  ],
  "eventType": [
    "processEvent"
  ]

```

What i would like to have is each detail populate field named by and have value of , for example:

```
 <detail>
  <name>pid</name>
  <value>1188</value>
 </detail>

```

to produce field: pid: 1188

but i'm not sure how to achieve this?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 7, 2019, 1:21pm UTC](https://discuss.elastic.co/t/parsing-nested-xml-with-logstash/202530/2 "2019-10-07T13:21:51Z")

</div>

You would need to use a ruby filter. Something like [this](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/12).

---

<div class="post-metadata">

**Author:** ![jsj](https://avatars.discourse-cdn.com/v4/letter/j/7ea924/32.png) [@jsj](https://discuss.elastic.co/u/jsj)\
**Post date:** [October 7, 2019, 1:56pm UTC](https://discuss.elastic.co/t/parsing-nested-xml-with-logstash/202530/3 "2019-10-07T13:56:59Z")

</div>

Okay,

So how do i point the field agentevent.details to json filter?

I tried

```
json {
    source => "agentevent.details"
}

```

But this gives an error.

In the thread you linked you had this field created but how can i access this with XML? If use xpath to create fields before mutate, i will have all the names and values in those fields;

mutate { add\_field =\> { "%{[Request][Headers][Name]}" =\> "%{[Request][Headers][Value]}" } }

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 7, 2019, 4:12pm UTC](https://discuss.elastic.co/t/parsing-nested-xml-with-logstash/202530/4 "2019-10-07T16:12:19Z")

</div>

> [@jsj](#):
>
> source =\> "agentevent.details"

That should be [agentevent][details]

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 4, 2019, 4:12pm UTC](https://discuss.elastic.co/t/parsing-nested-xml-with-logstash/202530/5 "2019-11-04T16:12:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.