Iteration in Logstash


(Jason Christopher Tan) #1

I have a string data in Logstash that has been parsed as that looks like

"id": "0,3,10"
"value": "100,200,300"

I would like to match the individual "id" to the corresponding "value" with respect to their order in the string.

I want the field to look like

"id0": "100"
"id3": "200"
"id10": "300"

As of now, I have used the mutate field to separate the array into its individual values. I used the following code:

mutate {
   add_field => {"id%{[id][0]}" => "%{[value][0]}"}
   add_field => {"id%{[id][1]}" => "%{[value][1]}"}
   add_field => {"id%{[id][2]}" => "%{[value][2]}"}
}

This code works fine fine. However, there are a varying number of values in the "id" field. Ie, some "id" field does not have id3, but some have an additional id4. However, for each id, there will definitely be a corresponding value in the "value" field. It will be difficult for me to make multiple if statements to fully separate all the possible combinations of different "id" values and its corresponding "value" values.

I heard that if I want make loop in Logstash, I have to use ruby filter. Unfortunately I'm not familiar with ruby

Thanks for any help. I really appreciate it.


Iteration/Ruby help in Logstash
Split array fields in logstash
어떤 필터를 사용해야 하나요?
(Magnus Bäck) #2
filter {
  ruby {
    code => '
      ids = event["id"].split(",")
      values = event["value"].split(",")
      if ids.length == values.length
        ids.each_index { |i| event["id#{ids[i]}"] = values[i] }
      end
    '
  }
}

How can I split fields
(Jason Christopher Tan) #3

Your code solved it! Thanks so much for your help :smile:


(Grigory Shamov) #4

Hi Magnus,

Is there a way to iterate over event fields with new LS5 event syntax in the similar way?


(Magnus Bäck) #5

Is there a way to iterate over event fields with new LS5 event syntax in the similar way?

Should work the same, just use event.get('foo') instead of event['foo'].


(system) #6