# Iterate parsing json in Logstash
**URL:** https://discuss.elastic.co/t/iterate-parsing-json-in-logstash/249749
**Category:** Logstash
**Created:** [September 24, 2020, 6:56am UTC](https://discuss.elastic.co/t/iterate-parsing-json-in-logstash/249749 "2020-09-24T06:56:28Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@Sivajanani](https://discuss.elastic.co/u/Sivajanani)
#### Post date: [September 24, 2020, 6:56am UTC](https://discuss.elastic.co/t/iterate-parsing-json-in-logstash/249749/1 "2020-09-24T06:56:29Z")
Hello All,
My Json field looks like below,
{"group" : [{"name":"group1","tag":[{"name":"tag1"},{"name":"tag2"},{"name":"tag3"},{"name":"tag4"}]}],"association" : }
And my config file having mutate filter is as follows,
mutate{
add\_field =\> {
"group\_name" =\> "%{[group][name]}"
"tag1" =\> "%{[group][tag][0][name]}"
"tag2" =\> "%{[group][tag][1][name]}"
"tag3" =\> "%{[group][tag][2][name]}"
"tag4" =\> "%{[group][tag][4][name]}"
}
}
This code works fine . However, there are a varying number of values in the "name" field. Ie, some json doesn't have 4 set of name:tagn inside tag and some have additional 5 or 6 set of name:tagn inside tag field. However, for each name, there will definitely be a corresponding tagn in the "value" field.
So this makes me difficult to automate ingestion and everytime I have to make changes in config file.
I heard we can solve it using ruby code. But I am a starter and I do not know much about ruby.
Can anyone help me iterating this based on number of sets of name:tagn available inside tag array using ruby.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 24, 2020, 2:13pm UTC](https://discuss.elastic.co/t/iterate-parsing-json-in-logstash/249749/2 "2020-09-24T14:13:35Z")
You could try
```
ruby {
code => '
a = event.get("[group][0][tag]")
a.each_index { |x|
event.set("tag#{x+1}", a[x]["name"])
}
'
}
```
---
### Author:  [@Sivajanani](https://discuss.elastic.co/u/Sivajanani)
#### Post date: [September 26, 2020, 4:05am UTC](https://discuss.elastic.co/t/iterate-parsing-json-in-logstash/249749/3 "2020-09-26T04:05:08Z")
HI,
Thanks for the reply.
I think this will help me iterating. When I am iterating I have to ingest these values to a particular meta.
Currently I tried something like below,
mutate{
add\_field =\> {
"group\_name" =\> "%{[group][name]}"
"test" =\> "%{[group][tag]}"
"tags" =\> ""
"tag" =\> "%{[group][tag][0][name]}"
}
}
ruby {
code =\> '
a = event.get("[group][0][tag]")
a.each\_index { |x|
event.set("tag#{x+1}", a["name"])
}
'
}
And my result is
{
"group\_name" =\> "group1",
"test" =\> "{name=tag1},{name=tag2},{name=tag3},{name=tag4},
"tag" =\> "tag1",
"tags" =\> [
[0] "",
[1] "\_rubyexception"
]
}
How can I overcome this and put these values to a particular meta.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 26, 2020, 2:34pm UTC](https://discuss.elastic.co/t/iterate-parsing-json-in-logstash/249749/4 "2020-09-26T14:34:36Z")
Please edit your post, select your configuration, and click on \ in the tool bar above the edit pane.
Also, what ruby exception does logstash log?
---
### Author:  [@Sivajanani](https://discuss.elastic.co/u/Sivajanani)
#### Post date: [September 28, 2020, 1:20pm UTC](https://discuss.elastic.co/t/iterate-parsing-json-in-logstash/249749/5 "2020-09-28T13:20:29Z")
Hello @Badger,
Sorry that I could not edit the previous post.
The ruby exception is as below,
`ERROR][logstash.filters.ruby][test] Ruby exception occurred: undefined method each_index' for nil:NilClass`
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 28, 2020, 1:44pm UTC](https://discuss.elastic.co/t/iterate-parsing-json-in-logstash/249749/6 "2020-09-28T13:44:12Z")
> [@Sivajanani](#):
>
> `Ruby exception occurred: undefined method each_index' for nil:NilClass`
That means [group][0][tag] does not exist. You could test for that in the ruby filter
```
ruby {
code => '
a = event.get("[group][0][tag]")
if a
a.each_index { |x|
event.set("tag#{x+1}", a[x]["name"])
}
end
'
}
```
---
### Author:  [@Sivajanani](https://discuss.elastic.co/u/Sivajanani)
#### Post date: [September 28, 2020, 2:24pm UTC](https://discuss.elastic.co/t/iterate-parsing-json-in-logstash/249749/7 "2020-09-28T14:24:37Z")
Hello @Badger,
Thanks for that. It really worked. I tried something below and it is getting results in the format of tag1:name1,tag2:name2
> ```
> mutate{
> add_field => {
> "group_name" => "%{[group][name]}"
> }
> }
> ruby {
> code => '
> a = event.get("[group][tag]")
> a.each_index { |x|
> event.set("tag#{x+1}", a[x]["name"])
> }
> '
> }
> }
> prune {
> whitelist_names => ["group_name","tag"]
> }
> fingerprint {
> method => "SHA256"
> source => ["group_name","tag"]
> target => "[@metadata][generate_id]"
> }
>
> ```
But since I have given prune and fingerprint filters it is filtering the tag1,tag2... fields. Can I specify something common there like tag\* or something which will take all the field starting with tag.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 28, 2020, 3:08pm UTC](https://discuss.elastic.co/t/iterate-parsing-json-in-logstash/249749/8 "2020-09-28T15:08:31Z")
The prune filter whitelist\_names option takes regexps. So "tag" will match "tag1" and "maytag".
The fingerprint you could do using something like
```
ruby {
code => '
a = event.get("[group][tag]")
if a
s = ""
a.each_index { |x|
event.set("tag#{x+1}", a[x]["name"])
s += ":" + a[x]["name"]
}
event.set("[@metadata][tags]", s)
'
}
fingerprint {
method => "SHA256"
source => ["group_name","[@metadata][tags]"]
target => "[@metadata][generate_id]"
}
```
That assumes the [group][tag] array is ordered. If not you will have to change a.each\_index to a.sort.each\_index
---
### Author:  [@Sivajanani](https://discuss.elastic.co/u/Sivajanani)
#### Post date: [September 29, 2020, 10:29am UTC](https://discuss.elastic.co/t/iterate-parsing-json-in-logstash/249749/9 "2020-09-29T10:29:07Z")
Hello @Badger
I have tried like below as suggested by you
> mutate{
> add\_field =\> {
> "group\_name" =\> "%{[group][name]}"
> }
> }
> ruby {
> code =\> '
> a = event.get("[group][tag]")
> if a
> s = ""
> a.each\_index { |x|
> event.set("tag#{x+1}", a["name"])
> s += ":" + a["name"]
> }
> event.set("[@metadata][tags]", s)
> '
> }
> prune {
> whitelist\_names =\> ["group\_name","tags"]
> }
> fingerprint {
> method =\> "SHA256"
> source =\> ["group\_name","[@metadata][tags]"]
> target =\> "[@metadata][generate\_id]"
> }
> I am getting the following error in ruby which I could not figure out,
> `[ERROR][org.logstash.Logstash] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SyntaxError) (ruby filter code):12: syntax error, unexpected end-of-file`
Please suggest.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 29, 2020, 2:49pm UTC](https://discuss.elastic.co/t/iterate-parsing-json-in-logstash/249749/10 "2020-09-29T14:49:23Z")
> [@Sivajanani](#):
>
> syntax error, unexpected end-of-file
Change
```
event.set("[@metadata][tags]", s)
'
```
to
```
event.set("[@metadata][tags]", s)
end
'
```
That closes out the 'if a' that I added to avoid the
```
undefined method each_index' for nil:NilClass
```
---
### Author:  [@Sivajanani](https://discuss.elastic.co/u/Sivajanani)
#### Post date: [September 30, 2020, 5:36am UTC](https://discuss.elastic.co/t/iterate-parsing-json-in-logstash/249749/11 "2020-09-30T05:36:08Z")
Hello @Badger,
I tried below adding and
> mutate{
> add\_field =\> {
> "group\_name" =\> "%{[group][name]}"
> }
> }
> ruby {
> code =\> '
> a = event.get("[group][tag]")
> if a
> s = ""
> a.each\_index { |x|
> event.set("tag#{x+1}", a["name"])
> s += ":" + a["name"]
> }
>
> ```
> event.set("[@metadata][tags]", s)
> end
> '
> }
>
> ```
>
> prune {
> whitelist\_names =\> ["group\_name","tags"]
> }
> fingerprint {
> method =\> "SHA256"
> source =\> ["group\_name","[@metadata][tags]"]
> target =\> "[@metadata][generate\_id]"
> }
But tags field itself is not created and the output is as follows,
`{ "group_name" => "name1" }`
Kindly suggest.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 30, 2020, 1:02pm UTC](https://discuss.elastic.co/t/iterate-parsing-json-in-logstash/249749/12 "2020-09-30T13:02:16Z")
> [@Sivajanani](#):
>
> whitelist\_names =\> ["group\_name","tags"]
That should be "tag", not "tags". Your field names are tag1, tag2 etc. and they do not start with "tags".
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 28, 2020, 1:02pm UTC](https://discuss.elastic.co/t/iterate-parsing-json-in-logstash/249749/13 "2020-10-28T13:02:29Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.