Split json array and apply filter


(Michael T.) #1

Hi,

I'm retrieving a Json Array as Input and would like to know how it would be possible to split it using the split filter in order to send each new item as a single output ?
In addition, the other filters should be applied to the new event raised, would it be possible to do it only by properly ordering the filter block ?

Example : Assuming I have the following input :

{ "results" : [
{"res1" : { "min": 1, "max": 2}},
{"res2" : { "min": 0, "max": 8}},
{"res3" : { "min": 4, "max": 6}},
{"res4" : { "min": 6, "max": 9}}
] }

The goal would be to split the results array into single output. But before that I would like to add to each resX a new filed (e.g. timestamp) using mutate and add_filed filters.

Apparently the pull request logstash-filter-split, should enable split of array but I didn't found the syntax to make it work. Also as described in the issue 2131 , the workaround is to use a ruby filter works but in this case the other filter are not applied to the split items.


(Michael T.) #2

found the solution :

  • using the split plugin to split the array :

split {
field => "results"
}

  • with the newly released 1.5.0, the filter are automatically applied to the split item automatically (wether this is due to filter order or internal filtering schedule, I don't know)

#3

Michael, thanks for the answer! I'm experiencing the same problem. Can you please clarify the second point?

What do you mean by "the filter are automatically applied to the split item automatically"?
Which filters, exactly, are applied to the split item automatically?


(system) #4