# Loop through and translate nested object **URL:** \ **Category:** Logstash\ **Created:** [March 4, 2019, 2:40pm UTC](https://discuss.elastic.co/t/loop-through-and-translate-nested-object/170742 "2019-03-04T14:40:53Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Ant](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ant/32/33267_2.png) [@Ant](https://discuss.elastic.co/u/Ant)\ **Post date:** [March 4, 2019, 2:40pm UTC](https://discuss.elastic.co/t/loop-through-and-translate-nested-object/170742/1 "2019-03-04T14:40:53Z")
Hi, I'm pretty sure I need to use ruby to achive this but I don't really know anything about it. I've done a translation with logstash already where I enrich an ID field that occurs once per document. What I need to do now though is add a name supplier to a nested array of data that could contain 1 or many repeating entries. For example: ```auto "customerID" : 123456, "customerName": "John smith", "Orders" : [ { "orderId" : "123456", "dateTimeOrdered" : "2019-02-01T00:07:39Z", "orderedItems" : [ { "objectType" : 0, "productId" : 1536 }, { "objectType" : 0, "productId" : 1529 }, { "objectType" : 0, "productId" : 1490 }, { "objectType" : 0, "productId" : 1535 } ] } ] ``` And using translation filter with records that look like '1536' : '{"name": "red ball", "supplier" : "Fred"}' '1529' : '{"name" : "yellow rocket","supplier": "Steve"}, '1490' : '{"name" : "fire truck","supplier" : "Jim Bob"}' '1535# : '{"name" : "squad car", "supplier" : "Jim Bob"}' Get logstash to use an elasticsearch pulgin to collect that data, a tranlation filter using the above dictionary file to loop through each of the orders and then use the elasticserach output pluging to update the docuemtent to look like. ```auto "customerID" : 123456, "customerName": "John smith", "Orders" : [ { "orderId" : "123456", "dateTimeOrdered" : "2019-02-01T00:07:39Z", "orderedItems" : [ { "objectType" : 0, "productId" : 1536, "name" : "red ball", "supplier" : "Fred" }, { "objectType" : 0, "productId" : 1529, "name" : "yellow rocket", "supplier": "Steve" }, { "objectType" : 0, "productId" : 1490, "name" : "fire truck", "supplier" : "Jim Bob" }, { "objectType" : 0, "productId" : 1535, "name" : "squad car" "supplier" : "Jim Bob" } ] } ] ``` There would be other fields in the docuement but I've omited them for the sake of brevity. I feel ruby comes into this somewhere but I don't know enough about it to understand where to start with it to make headway on this. Many thanks for your help in advance Kind regards Ant ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [March 4, 2019, 3:06pm UTC](https://discuss.elastic.co/t/loop-through-and-translate-nested-object/170742/2 "2019-03-04T15:06:23Z")
Take a look at the iterate\_on option to translate. ``` translate { iterate_on => "[Orders][0][orderedItems]" field => "productId" destination => "foo" dictionary_path => "/home/user/foo.yml" } ``` will get you ``` "orderedItems" => [ [0] { "objectType" => 0, "productId" => 1536, "foo" => "{\"name\": \"red ball\", \"supplier\" : \"Fred\"}" }, [1] { "objectType" => 0, "productId" => 1529, "foo" => "{\"name\" : \"yellow rocket\",\"supplier\": \"Steve\"}" }, ``` Note that if your yml file entries did not have single quotes on the RHS and just looked like ``` '1535' : {"name" : "squad car", "supplier" : "Jim Bob"} ``` Then you would instead get ``` "orderedItems" => [ [0] { "objectType" => 0, "productId" => 1536, "foo" => { "name" => "red ball", "supplier" => "Fred" } }, [1] { "objectType" => 0, "productId" => 1529, "foo" => { "name" => "yellow rocket", "supplier" => "Steve" } }, ``` In either case a fairly simple ruby filter can be used to adjust the field names. ---
**Author:** ![Ant](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ant/32/33267_2.png) [@Ant](https://discuss.elastic.co/u/Ant)\ **Post date:** [March 4, 2019, 5:33pm UTC](https://discuss.elastic.co/t/loop-through-and-translate-nested-object/170742/3 "2019-03-04T17:33:50Z")
Perfect, had an issue as the documents had been assigned a type that wasn't doc so when it tried to write them as it writes as type "doc" it failed, I re-indexed a sample index with a type of "doc" to proof the process though and it worked a treat. when I was doing translations without the itteration I used the result in single quotes as I could then do ```auto json { source => "translation" remove_field => ["translation"] } ``` on the filter stage which would allow ```auto { "objectType" : 0, "productId" : 1490, "name" : "fire truck", "supplier" : "Jim Bob" } ``` rather than ```auto { "objectType" : 0, "productId" : 1490, "foo" : { "name" : "fire truck", "supplier" : "Jim Bob" } } ``` the json filter doesn't look to support itteration in the same way though so I can't do that in this situation. I realise I could create 2 dictionarys and then run the filter once for name and once for supplier to get the desired result but is there a more eligant way? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [March 4, 2019, 5:45pm UTC](https://discuss.elastic.co/t/loop-through-and-translate-nested-object/170742/4 "2019-03-04T17:45:24Z")
> [@Ant](#): > > the json filter doesn't look to support itteration in the same way though so I can't do that in this situation. I realise I could create 2 dictionarys and then run the filter once for name and once for supplier to get the desired result but is there a more eligant way? I cannot think of one 🙂 ---
**Author:** ![Ant](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ant/32/33267_2.png) [@Ant](https://discuss.elastic.co/u/Ant)\ **Post date:** [March 4, 2019, 5:51pm UTC](https://discuss.elastic.co/t/loop-through-and-translate-nested-object/170742/5 "2019-03-04T17:51:16Z")
It was worth checking 🙂 I am very greatful for your insights though! ---
**Author:** ![Ant](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ant/32/33267_2.png) [@Ant](https://discuss.elastic.co/u/Ant)\ **Post date:** [March 5, 2019, 4:05pm UTC](https://discuss.elastic.co/t/loop-through-and-translate-nested-object/170742/6 "2019-03-05T16:05:16Z")
> [@Badger](#): > > iterate\_on =\> "[Orders][0][orderedItems]" Just realised something, there are times when there is more than one order against a customer but the [0] would seem to limit it to only the first order in the array. I've tried `iterate_on => "[Orders][orderedItems]"` & `iterate_on => "[Orders][][orderedItems]"` but neither seem to work and I can't find the required syntax online, you wouldn't happen to know what magical character unlocks the behaviour I need? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [March 5, 2019, 5:33pm UTC](https://discuss.elastic.co/t/loop-through-and-translate-nested-object/170742/7 "2019-03-05T17:33:09Z")
> [@Ant](#): > > but neither seem to work and I can't find the required syntax online, you wouldn't happen to know what magical character unlocks the behaviour I need? I do not see any code in the filter that would allow it to iterate a pair of nested arrays. Depending on what you need to do with the events subsequently it might be viable to use a split filter to separate the orders into multiple events, at which point you could use translate as shown above. If you have a known number of orderedItems (or a limit to them) then you could grit your teeth and ``` translate { iterate_on => "[Orders]" field => "[orderedItems][0][productId]" destination => "[orderedItems][0][foo]" [...] } translate { iterate_on => "[Orders]" field => "[orderedItems][1][productId]" destination => "[orderedItems][1][foo]" [...] } ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 2, 2019, 5:33pm UTC](https://discuss.elastic.co/t/loop-through-and-translate-nested-object/170742/8 "2019-04-02T17:33:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.