# Xml Parsing in logstash using xml & split plugins

**URL:** <https://discuss.elastic.co/t/xml-parsing-in-logstash-using-xml-split-plugins/300515>\
**Category:** Logstash\
**Created:** [March 24, 2022, 5:30am UTC](https://discuss.elastic.co/t/xml-parsing-in-logstash-using-xml-split-plugins/300515 "2022-03-24T05:30:50Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![manikandanid](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/manikandanid/32/91013_2.png) [@manikandanid](https://discuss.elastic.co/u/manikandanid)\
**Post date:** [March 24, 2022, 5:30am UTC](https://discuss.elastic.co/t/xml-parsing-in-logstash-using-xml-split-plugins/300515/1 "2022-03-24T05:30:50Z")

</div>

I have a xml which looks like

```auto
<Countries><Country><Name>India</Name><ISDCode>+91</ISDCode><Continent>Asia</Continent><geolocationinfo><lattitude>123129</lattitude><longititude>7890890</longititude></geolocationinfo></Country><Country><Name>Srilanka</Name><ISDCode>+94</ISDCode><Continent>Asia</Continent><geolocationinfo><lattitude>1212349</lattitude><longititude>123890</longititude></geolocationinfo></Country></Countries>

```

All the content of the xml are in single line and we use filebeat to collect the data from log file and send it to logstash.

We need to store this xml as two document in Elasticsearch

doc1:  
Name: India  
ISDCode: +91  
continent: Asia  
geolocationinfo.Lattitude: 123129  
geolocationinfo.longititude: 7890890

doc2:  
Name: Srilanka  
ISDCode: +94  
continent: Asia  
geolocationinfo.Lattitude: 1212349  
geolocationinfo.longititude: 123890

Following is the logstash.conf that we have

```auto
input {
  beats {
    port => 5044
  }
}
filter
{
	xml
	{
		source => "message"
		target => "xml_content"
		store_xml => true
		xpath => 
		[            
            "/Countries/Country/Name/text()", "Name",
			"/Countries/Country/ISDCode/text()", "ISDCode",
			"/Countries/Country/Continent/text()", "Continent",
			"/Countries/Country/geolocationinfo/lattitude/text()", "geolocationinfo.lattitude",
			"/Countries/Country/geolocationinfo/longititude/text()", "geolocationinfo.longititude"
		]
		
	}
	split{
		field => 'Name'
		}
	}
}

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "test"
  }
}

```

What we get is two documents but the document contains only Name column and other columns are missing. I tried adding add\_field also to split plugin but no luck.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 24, 2022, 4:47pm UTC](https://discuss.elastic.co/t/xml-parsing-in-logstash-using-xml-split-plugins/300515/2 "2022-03-24T16:47:59Z")

</div>

The output from the xml filter will be an event which has several arrays which each have two entries. I cannot think of clean way of creating an event from the first entry in each of five array fields, and another event from the second entry in each of those field other than using ruby. It would be simpler to do

```
    xml {
        source => "message"
        target => "[@metadata][xml]"
        store_xml => true
        remove_field => ["message"]
    }
    split { field => "[@metadata][xml][Country]" }
    mutate {
        add_field => {
            "Name" => "%{[@metadata][xml][Country][Name][0]}"
            "ISDCode" => "%{[@metadata][xml][Country][ISDCode][0]}"
            "continent" => "%{[@metadata][xml][Country][Continent][0]}"
            "geolocation.latitude" => "%{[@metadata][xml][Country][geolocationinfo][0][lattitude][0]}"
            "geolocation.longitude" => "%{[@metadata][xml][Country][geolocationinfo][0][longititude][0]}"
        }
    }

```

If you do the split on the Country array then it keeps all five fields together in the same event and you can just add\_field them into the right place.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 21, 2022, 4:48pm UTC](https://discuss.elastic.co/t/xml-parsing-in-logstash-using-xml-split-plugins/300515/3 "2022-04-21T16:48:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.