# Mutate Convert XML Multiple Layers

**URL:** <https://discuss.elastic.co/t/mutate-convert-xml-multiple-layers/177678>\
**Category:** Logstash\
**Created:** [April 19, 2019, 10:49pm UTC](https://discuss.elastic.co/t/mutate-convert-xml-multiple-layers/177678 "2019-04-19T22:49:12Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![MarkBeach](https://avatars.discourse-cdn.com/v4/letter/m/49beb7/32.png) [@MarkBeach](https://discuss.elastic.co/u/MarkBeach)\
**Post date:** [April 19, 2019, 10:49pm UTC](https://discuss.elastic.co/t/mutate-convert-xml-multiple-layers/177678/1 "2019-04-19T22:49:12Z")

</div>

I'm running into an issue trying to use mutate to convert and XML file. I can convert the first level, but if the XML file has multiple elements then they output as string.

**Logstash Configuration**

```auto
filter {
    if [fields][log] == "INSPECTION_AOI" {
        xml {
            source => "message"
            store_xml => true
            target => "Inspection"
            remove_field => [
                "message"
            ]
        }
        mutate {
            convert => {
                "[Inspection][CycleTime]" => "integer"
                "[Inspection][TotalFeatures]" => "integer"
                "[Inspection][TotalComponents]" => "integer"
                "[Inspection][LotSize]" => "integer"
                "[Inspection][Board][0][TotalFeatures][Value]" => "integer"
                "[Inspection][Board][0][TotalComponents][Value]" => "integer"
                "[Inspection][Board][0][Component][0][IncludedInCurrentVariant]" => "boolean"
                "[Inspection][Board][0][Component][0][TotalFeatures]" => "integer"
                "[Inspection][Board][0][Component][0][Feature][0][Inspection][0][Measurements][0][X][0][Value]" => "float"
                "[Inspection][Board][0][Component][0][Feature][0][Inspection][0][Measurements][0][X][0][Minimum]" => "float"
                "[Inspection][Board][0][Component][0][Feature][0][Inspection][0][Measurements][0][X][0][Maximum]" => "float"
                "[Inspection][Board][0][Component][0][Feature][0][Inspection][0][Measurements][0][X][0][Target]" => "float"
                "[Inspection][Board][0][Component][0][Feature][0][Inspection][0][Measurements][0][Y][0][Value]" => "float"
                "[Inspection][Board][0][Component][0][Feature][0][Inspection][0][Measurements][0][Y][0][Minimum]" => "float"
                "[Inspection][Board][0][Component][0][Feature][0][Inspection][0][Measurements][0][Y][0][Maximum]" => "float"
                "[Inspection][Board][0][Component][0][Feature][0][Inspection][0][Measurements][0][Y][0][Target]" => "float"
                "[Inspection][Board][0][Component][0][Feature][0][Inspection][0][Measurements][0][ConfidenceLevel][0][Value]" => "float"
                "[Inspection][Board][0][Component][0][Feature][0][Inspection][0][Measurements][0][ConfidenceLevel][0][Minimum]" => "float"
                "[Inspection][Board][0][Component][0][Feature][0][Inspection][0][Measurements][0][ConfidenceLevel][0][Maximum]" => "float"
                "[Inspection][Board][0][Component][0][Feature][0][Inspection][0][Measurements][0][ConfidenceLevel][0][Target]" => "float"
          }
        }
    }
}

output {
    if [fields][log] == "INSPECTION_AOI" {
        elasticsearch {
            hosts => "localhost"
            index => "inspection_aoi"
        }
        
        stdout { codec => rubydebug }
    }
}

```

**File 1 (XML)**

```auto
<Panel Id="1" Name="Panel" PanelStatus="Failed" StartTime="04/10/2019 09:09:07" EndTime="04/10/2019 09:11:12" CycleTime="4" TotalFeatures="7704" TotalComponents="7704" LotCode="TEST" LotSize="0" Barcode="10695 05 A07502 4718" Code="10695 05 A07502 4718" NG_Stacker_SlotNumber="">
  <Board Id="1" Name="Board1" BoardStatus="Failed" TotalFeatures="7704" TotalComponents="7704" Barcode="" Skip="false">
    <Component Id="6" Name="C1_D1" PartNumber="LED" Package="LED" Barcode="" IncludedInCurrentVariant="false" ComponentStatus="Passed" TotalFeatures="1">
      <Feature Id="7" Identifier="Body" FeatureStatus="Passed" FeatureInspectionInfo="" MeasurementValid="">
          <Inspection Identifier="Appearance" Type="AI2" status="Passed">
            <Measurements>
              <X Value="-0.006" Minimum="-0.079999999999886329" Maximum="0.0800000000001137" Target="0" />
              <Y Value="-0.006" Minimum="-0.079999999999886329" Maximum="0.0800000000001137" Target="0" />
              <ConfidenceLevel Value="-0.006" Minimum="-0.079999999999886329" Maximum="0.0800000000001137" Target="0" />
            </Measurements>
          </Inspection>
      </Feature>
    </Component>
  </Board>
</Panel>

```

**File 2 (XML)**

```auto
<Panel Id="1" Name="Panel" PanelStatus="Failed" StartTime="04/10/2019 09:09:07" EndTime="04/10/2019 09:11:12" CycleTime="4" TotalFeatures="7704" TotalComponents="7704" LotCode="TEST" LotSize="0" Barcode="10695 05 A07502 4718" Code="10695 05 A07502 4718" NG_Stacker_SlotNumber="">
  <Board Id="1" Name="Board1" BoardStatus="Failed" TotalFeatures="7704" TotalComponents="7704" Barcode="" Skip="false">
    <Component Id="6" Name="C1_D1" PartNumber="LED" Package="LED" Barcode="" IncludedInCurrentVariant="false" ComponentStatus="Passed" TotalFeatures="1">
      <Feature Id="7" Identifier="Body" FeatureStatus="Passed" FeatureInspectionInfo="" MeasurementValid="">
          <Inspection Identifier="Appearance" Type="AI2" status="Passed">
            <Measurements>
              <X Value="-0.006" Minimum="-0.079999999999886329" Maximum="0.0800000000001137" Target="0" />
              <Y Value="-0.006" Minimum="-0.079999999999886329" Maximum="0.0800000000001137" Target="0" />
              <ConfidenceLevel Value="-0.006" Minimum="-0.079999999999886329" Maximum="0.0800000000001137" Target="0" />
            </Measurements>
          </Inspection>
          <Inspection Identifier="Appearance" Type="AI2" status="Passed">
            <Measurements>
              <X Value="-0.006" Minimum="-0.079999999999886329" Maximum="0.0800000000001137" Target="0" />
              <Y Value="-0.006" Minimum="-0.079999999999886329" Maximum="0.0800000000001137" Target="0" />
              <ConfidenceLevel Value="-0.006" Minimum="-0.079999999999886329" Maximum="0.0800000000001137" Target="0" />
            </Measurements>
          </Inspection>
          <Inspection Identifier="Appearance" Type="AI2" status="Passed">
            <Measurements>
              <X Value="-0.006" Minimum="-0.079999999999886329" Maximum="0.0800000000001137" Target="0" />
              <Y Value="-0.006" Minimum="-0.079999999999886329" Maximum="0.0800000000001137" Target="0" />
              <ConfidenceLevel Value="-0.006" Minimum="-0.079999999999886329" Maximum="0.0800000000001137" Target="0" />
            </Measurements>
          </Inspection>
      </Feature>
    </Component>
  </Board>
</Panel>

```

**Result**  
File 1 - ConfidenceLevel, Y, X come in a FLOAT for index 0

File 2 - ConfidenceLevel, Y, X come in a FLOAT for index 0  
File 2 - ConfidenceLevel, Y, X come in a STRING for index 1  
File 2 - ConfidenceLevel, Y, X come in a STRING for index 2

I realize I'm only converting the first index of the XML in my logstash configuration. Is there a way to define a dynamic value for the index instead of hardcoding a [0].

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 20, 2019, 12:25am UTC](https://discuss.elastic.co/t/mutate-convert-xml-multiple-layers/177678/2 "2019-04-20T00:25:00Z")

</div>

> [@MarkBeach](#):
>
> I realize I'm only converting the first index of the XML in my logstash configuration. Is there a way to define a dynamic value for the index instead of hardcoding a [0].

I was going to suggest you use the 'force\_array =\> false' option on the xml filter. Then the inner Inspection field will be an array but Board, Component, and Feature will not. However I now realize that is a mistake, since it makes the first XML harder to parse.

You would use a ruby filter to handle the inner Inspection. This is an example of how to convert the second XML

```
    xml {
        source => "message"
        store_xml => true
        target => "Inspection"
        remove_field => ["message"]
        force_array => false
    }
    ruby {
        code => '
            replacementInspection = []
            i = event.get("[Inspection][Board][Component][Feature][Inspection]")
            i.each_index { |x|
                replacementMeasurements = {}
                r = i[x]
                m = r["Measurements"]
                m.each { |k, v|
                    h = {}
                    h["Value"] = v["Value"].to_f
                    h["Maximum"] = v["Maximum"].to_f
                    h["Minimum"] = v["Minimum"].to_f
                    h["Target"] = v["Target"].to_f
                    replacementMeasurements[k] = h
                }
                r["Measurements"] = replacementMeasurements
                replacementInspection << r
            }
            event.set("[Inspection][Board][Component][Feature][Inspection]", replacementInspection)
        '
    }

```

It needs to be re-written to handle 'force\_array =\> true' and then handle the first XML format But if you can do ruby (and if you do a lot of logstash it is worth learning) then it should help you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 18, 2019, 12:25am UTC](https://discuss.elastic.co/t/mutate-convert-xml-multiple-layers/177678/3 "2019-05-18T00:25:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.