# Urgent Help : How to Parse nested XML (Key value patterns) with multiple Parent children

**URL:** https://discuss.elastic.co/t/urgent-help-how-to-parse-nested-xml-key-value-patterns-with-multiple-parent-children/312546
**Category:** Logstash
**Created:** [August 21, 2022, 2:27pm UTC](https://discuss.elastic.co/t/urgent-help-how-to-parse-nested-xml-key-value-patterns-with-multiple-parent-children/312546 "2022-08-21T14:27:21Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![sonirajil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sonirajil/32/99235_2.png) [@sonirajil](https://discuss.elastic.co/u/sonirajil)
#### Post date: [August 21, 2022, 2:27pm UTC](https://discuss.elastic.co/t/urgent-help-how-to-parse-nested-xml-key-value-patterns-with-multiple-parent-children/312546/1 "2022-08-21T14:27:21Z")

</div>

Hello Community,  
I have minimal exp. in xml logstash parsing stuff.  
I have explored various discussions , one such which matches a bit to my case is : [[SOLVED] Split filter question a.k.a flatten json sub array - #10 by blastik](https://discuss.elastic.co/t/solved-split-filter-question-a-k-a-flatten-json-sub-array/130481/10) , which I tried but didn't work for me due to multiple hierarchy.  
I have problem parsing voice console data which looks like below :

```auto
<?xml version="1.0" encoding="UTF-8"?>
<terminalCommand serialNumber="6217015078" commandID="1261">
  <timestamp>2022-07-21 09:43:48 GMT-04:00</timestamp>
  <terminalState id="3">SLEEPING</terminalState>
  <command>
    <sendTerminalProperties commandVersion="2">
      <property name="multipleResidentTtsEnabled" value="true" />
      <group name="Bluetooth">
        <property name="discoverable" value="false" />
        <property name="scanner_supported" value="true" />
        <group name="PairHeadset">
          <property name="address" value="00:14:28:06:A3:66" />
          <property name="status" value="Searching" />
          <property name="type" value="Headset" />
        </group>
      </group>
      <group name="Network" />
      <group name="Battery">
        <property name="BattFirstUseDateUtc" value="1654228800" />
        <property name="BatteryHealthCode" value="0" />
        <property name="ChargeCapacity" value="8.902" />
        <property name="PartNumber" value="BT-901" />
        <property name="SerialNumber" value="351639132105" />
      </group>
    </sendTerminalProperties>
  </command>
</terminalCommand>

```

My logstash conf is :

```auto
input {
  beats {
        port => 50xx
        client_inactivity_timeout => 1200
        }
}
filter {
    grok {
                match => { "message" => "^%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:loglevel}\s+(\[)%{DATA:logger}(\])\s+%{DATA:command} (\|)%{DATA:output}?\s*$(?<stacktrace>(?m:.*))" }
        }
     if "<?xml" in [stacktrace]
      {
      xml {
    source => "stacktrace"
    target => "xmldata"
}
}

```

But due the nature of data it is , I get multiple values for single field per event , for example :  
_xmldata.command.sendTerminalProperties.group.name_ : **Bluetooth, Network, Battery**  
_xmldata.command.sendTerminalProperties.group.property.name_ : **discoverable, discoverable\_modifiable, display\_supported, enabled, enabled\_modifiable, headset\_manual\_pairing\_enabled, headset\_manual\_pairing\_modifiable, headset\_supported, printer\_supported, scanner\_supported, BattFirstUseDateUtc, BatteryHealthCode, ChargeCapacity, ChargeCycles, DesignCapacity, PartNumber, RunTime, SerialNumber**  
_xmldata.command.sendTerminalProperties.group.property.value_ : **false, false, true, true, true, true, true, true, true, 1645660800, 0, 9.719, 35, 10.73, 730051, 0, 3521401**

My expected output should be like  
xmldata.command.sendTerminalProperties.group.name : Battery  
xmldata.command.sendTerminalProperties.group.property.name : BatteryHealthCode  
xmldata.command.sendTerminalProperties.group.property.value : 2  
xmldata.command.sendTerminalProperties.group.name : Battery  
xmldata.command.sendTerminalProperties.group.property.name : ChargeCapacity  
xmldata.command.sendTerminalProperties.group.property.value : 2

so that I can use particular values for dashboard.  
Request quick advice.  
Please let me know if any additional info required from me.  
Thanks

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 21, 2022, 9:48pm UTC](https://discuss.elastic.co/t/urgent-help-how-to-parse-nested-xml-key-value-patterns-with-multiple-parent-children/312546/2 "2022-08-21T21:48:08Z")

</div>

> [@sonirajil](#):
>
> xmldata.command.sendTerminalProperties.group.property.name : BatteryHealthCode  
> xmldata.command.sendTerminalProperties.group.property.name : ChargeCapacity

The only way to have two objects with the same name is to make the object an array. I would significantly rearrange the data using ruby to expand all the name/value pairs.

```
    xml { source => "message" force_array => false target => "xmldata" }
    ruby {
        init => '
            def doSomething(object, name, keys, event)
#puts "doSomething called for #{name}"
                if object
                    if object.kind_of?(Hash) and object != {}
                        if name =~ /\[property\]$/
                            event.set(name, { object["name"] => object["value"] })
                        else
                            object.each { |k, v| doSomething(v, "#{name}[#{k}]", keys, event) }
                        end
                    elsif object.kind_of?(Array) and object != []
                        if name =~ /\[property\]$/
                            h = {}
                            object.each_index { |i|
                                h[object[i]["name"]] = object[i]["value"]
                            }
                            event.set(name, h)
                        else
                            object.each_index { |i|
                                doSomething(object[i], "#{name}[#{i}]", keys, event)
                            }
                        end
                    else
#puts "working on #{name}"
                    end
                end
            end
        '
        code => '
            xml = event.get("xmldata")
            if xml
                doSomething(xml, "[xmldata]", @field, event)
            end
        '
    }

```

This will get you

```
        "sendTerminalProperties" => {
                  "property" => {
                "multipleResidentTtsEnabled" => "true"
            },
            "commandVersion" => "2",
                     "group" => [
                [0] {
                        "name" => "Bluetooth",
                    "property" => {
                             "discoverable" => "false",
                        "scanner_supported" => "true"
                    },
                       "group" => {
                        "property" => {
                             "status" => "Searching",
                               "type" => "Headset",
                            "address" => "00:14:28:06:A3:66"
                        },
                            "name" => "PairHeadset"
                    }
                },
                [1] {
                    "name" => "Network"
                },

```

etc. I would consider further reorganizing the outer group object but that is a lot more code to write.

---

<div class="post-metadata">

### Author: ![sonirajil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sonirajil/32/99235_2.png) [@sonirajil](https://discuss.elastic.co/u/sonirajil)
#### Post date: [August 22, 2022, 5:05am UTC](https://discuss.elastic.co/t/urgent-help-how-to-parse-nested-xml-key-value-patterns-with-multiple-parent-children/312546/3 "2022-08-22T05:05:01Z")

</div>

Thanks for your quick response Badger , in interest of time in case If I need just specific fields with corresponding values instead of flattening whole hierarchy e.g. specifically  
BatteryHealthCode  
ChargeCapacity  
PartNumber  
SerialNumber  
from parent xmldata.command.sendTerminalProperties.group.name : **Battery**.  
Is there any other way like split or something you can advise .  
I tried split which you suggested in below (might be ,my code is not correct)

> [@Possible to Parse Nested XML Data structures and combine parent with child](https://discuss.elastic.co/t/possible-to-parse-nested-xml-data-structures-and-combine-parent-with-child/167854/3):
>
> I am starting with that. I can see how to do it if i only ever have 1 parent but when there are muliple parent / children blocks thats what i cant see how to break it down.

```auto
filter {
    grok {
                match => { "message" => "^%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:loglevel}\s+(\[)%{DATA:logger}(\])\s+%{DATA:command} (\|)%{DATA:output}?\s*$(?<stacktrace>(?m:.*))" }
        }
     if "<?xml" in [stacktrace]
      {
      xml {
    source => "stacktrace"
    target => "xmldata"
        }
}
         split { field => "[xmldata][command][sendTerminalProperties][group][name]" }
         split { field => "[xmldata][command][sendTerminalProperties][property][name]" }
         split { field => "[xmldata][command][sendTerminalProperties][property][value]" }

}

```

Request advice.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [August 22, 2022, 4:16pm UTC](https://discuss.elastic.co/t/urgent-help-how-to-parse-nested-xml-key-value-patterns-with-multiple-parent-children/312546/4 "2022-08-22T16:16:03Z")

</div>

If the structure of the XML is fixed and you only need a few elements then you could use

```
    mutate {
        add_field => {
            "BatteryHealthCode" => "%{[xmldata][command][sendTerminalProperties][group][2][property][1][value]}"
            "ChargeCapacity" => "%{[xmldata][command][sendTerminalProperties][group][2][property][2][value]}"
        }
    }

```

---

<div class="post-metadata">

### Author: ![sonirajil](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sonirajil/32/99235_2.png) [@sonirajil](https://discuss.elastic.co/u/sonirajil)
#### Post date: [September 6, 2022, 7:30am UTC](https://discuss.elastic.co/t/urgent-help-how-to-parse-nested-xml-key-value-patterns-with-multiple-parent-children/312546/5 "2022-09-06T07:30:14Z")

</div>

We used the similar concept and could parse , Thanks a lot Badger , you have been a great help !!! Keep rocking ! 🙂

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 4, 2022, 7:31am UTC](https://discuss.elastic.co/t/urgent-help-how-to-parse-nested-xml-key-value-patterns-with-multiple-parent-children/312546/6 "2022-10-04T07:31:04Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
