# Iteration and loop in logstash

**URL:** <https://discuss.elastic.co/t/iteration-and-loop-in-logstash/80485>\
**Category:** Logstash\
**Created:** [March 29, 2017, 1:18pm UTC](https://discuss.elastic.co/t/iteration-and-loop-in-logstash/80485 "2017-03-29T13:18:20Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mohamed](https://avatars.discourse-cdn.com/v4/letter/m/d78d45/32.png) [@Mohamed](https://discuss.elastic.co/u/Mohamed)\
**Post date:** [March 29, 2017, 1:18pm UTC](https://discuss.elastic.co/t/iteration-and-loop-in-logstash/80485/1 "2017-03-29T13:18:20Z")

</div>

Hello,

I have a list data in Logstash that has been parsed as that looks like

`"Attribute": [`  
`"<Book><Name>N1</Name><Value>Clement</Value></Book>",`  
`"<Book><Name>N2</Name><Value>Bundle Name</Value></Book>"`  
`],`

I want to loop over that list to generate the following data in ouput:

"book1":{  
"key": "N1",  
"value":"Clement"  
},  
"book2":{  
"key": "N2",  
"value":"Bundle Name"  
}

Is it possible to do that?  
Thanks for any help. I really appreciate it.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 29, 2017, 1:38pm UTC](https://discuss.elastic.co/t/iteration-and-loop-in-logstash/80485/2 "2017-03-29T13:38:28Z")

</div>

The xml filter requires the source field to be a string or a single-element array so it won't help you. I think your best bet is to write a small snippet of Ruby and put in a ruby filter. I don't know if there's a gem for parsing XML that you can use, though. Worst-case you'll have to use regexps. Obviously a bad option for parsing XML but if the form of the document is well-known it might be good enough.

Writing a custom filter plugin would of course work too.

---

<div class="post-metadata">

**Author:** ![Mohamed](https://avatars.discourse-cdn.com/v4/letter/m/d78d45/32.png) [@Mohamed](https://discuss.elastic.co/u/Mohamed)\
**Post date:** [March 29, 2017, 4:17pm UTC](https://discuss.elastic.co/t/iteration-and-loop-in-logstash/80485/3 "2017-03-29T16:17:01Z")

</div>

I've tried the split plugin, but I found a bizarre behavior...

```
filter {
  split {
    field => "Attribute"
    target => "book_bundle"
    periodic_flush => "false"
  }
}

```

Which will take my source event, clone it into N new events, where N is the number of members of the Attribute array. The array member will be put into the book\_bundle field.  
and then I added mutate with gsub option to transform book\_bundle field and place it into a book1 .. bookN fields...

The problem here, when my logstash process is running, I find only the first element transformed like I want.. and then when I stop the process I find the rest of array's elements in the output file!

I can't understand this behavior.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [March 30, 2017, 5:17am UTC](https://discuss.elastic.co/t/iteration-and-loop-in-logstash/80485/4 "2017-03-30T05:17:33Z")

</div>

Why are you disabling `periodic_flush`? The behavior you're describing sounds like what might happen when you disable periodic flushing.

---

<div class="post-metadata">

**Author:** ![Mohamed](https://avatars.discourse-cdn.com/v4/letter/m/d78d45/32.png) [@Mohamed](https://discuss.elastic.co/u/Mohamed)\
**Post date:** [March 30, 2017, 8:05am UTC](https://discuss.elastic.co/t/iteration-and-loop-in-logstash/80485/5 "2017-03-30T08:05:02Z")

</div>

It works with ES output.. Thank you very much.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 27, 2017, 8:05am UTC](https://discuss.elastic.co/t/iteration-and-loop-in-logstash/80485/6 "2017-04-27T08:05:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.