# Im stuck at formatting data while trying to send xml file to Elasticsearch

**URL:** <https://discuss.elastic.co/t/im-stuck-at-formatting-data-while-trying-to-send-xml-file-to-elasticsearch/278002>\
**Category:** Logstash\
**Created:** [July 6, 2021, 10:25pm UTC](https://discuss.elastic.co/t/im-stuck-at-formatting-data-while-trying-to-send-xml-file-to-elasticsearch/278002 "2021-07-06T22:25:24Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sourtoast](https://avatars.discourse-cdn.com/v4/letter/s/ebca7d/32.png) [@Sourtoast](https://discuss.elastic.co/u/Sourtoast)\
**Post date:** [July 6, 2021, 10:25pm UTC](https://discuss.elastic.co/t/im-stuck-at-formatting-data-while-trying-to-send-xml-file-to-elasticsearch/278002/1 "2021-07-06T22:25:24Z")

</div>

I'm trying to add data from xml to Elasticsearch but I want only specific fields with its names changed.  
I'm using mutate filter to map xml fields to my own fields. My problem is with photos

```auto
input {
	http_poller {
		urls => {
			test => "https://super-secret-url.com"
		}
		schedule => { cron => "* * * * * UTC" }
		codec => "plain"
	}
}

filter {
	# Transform XML file to datastructure with field named 'source_file_products' containing array of XML strings for every product
	xml {
		source => "message"
		force_array => false
		xpath => ["/dane/produkty/p", "source_file_products"]
		store_xml => false
		remove_field => "message"
	}
	# Split array of XML strings as different events
	split { field => "source_file_products" }
	# Parse XML to datastructure
	xml {
		source => "source_file_products"
		force_array => false
		target => "source_file_product"
		remove_field => ["source_file_products", "@timestamp", "@version"]
	}
	# Get only relevant fields and map them respectively
	mutate {
		add_field => { 
			"_id" => "%{[source_file_product][id]}"
			"name" => "%{[source_file_product][nazwa]}"
			"price" => "%{[source_file_product][cena]}"
			"shortDescription" => "%{[source_file_product][html_description]}"
			"ean" => "%{[source_file_product][kod_ean]}"
			"mpn" => "%{[source_file_product][kod_producenta]}"
			"photos" => "%{[source_file_product][photos][0][url]}"
		}
		remove_field => "source_file_product"
	}
}
output {
	file {
		path => "./test.ndjson"
		codec => "json_lines"
	}
}

```

Data structure of [source\_file\_product][photos] looks like this:

```auto
"photos" => [
	[0] {
		"foo" => "bar",
		"attr1" => "1",
		"url" => "First link to a photo"
	},
	[1] {
		"url" => "Second link to a photo"
	},
	[2] {
		"more" => "clutter"
		"url" => "Third link to a photo"
	},
	[3] {
		"url" => "Fourth link to a photo"
	}
]

```

But I need:

```auto
"photos" => [
	"First link to a photo",
	"Second link to a photo",
	"Third link to a photo",
	"Fourth link to a photo"
]

```

The issue is that I don't know how many of photos will be in this array

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 6, 2021, 11:23pm UTC](https://discuss.elastic.co/t/im-stuck-at-formatting-data-while-trying-to-send-xml-file-to-elasticsearch/278002/2 "2021-07-06T23:23:30Z")

</div>

You can do that in ruby. I have not tested it but something like

```
ruby {
    code => '
        p = event.get("[source_file_product][photos]")
        if p.is_a? Array
            newP = []
            p.each_index { |x|
                newP << x["url"]
            }
            event.set("[source_file_product][photos]", newP)
    '
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 3, 2021, 11:24pm UTC](https://discuss.elastic.co/t/im-stuck-at-formatting-data-while-trying-to-send-xml-file-to-elasticsearch/278002/3 "2021-08-03T23:24:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.