Using logstash+http_poller to loop through JSON structure, run 2nd poller

[dorj1234 (Dor Juravski)]

Hello,
I have an http_poller working and getting me an array of JSON structures.
Is it possible in logstash to loop through one of the fields and for each one run a second http_poller?

For example, let's say my structure has a "City" field. I would like to run a second REST API for each city. Can it be done without hitting ruby coding? or if I must use ruby - can you point me in the right direction on how ruby does JSON parsing?
Or for ruby, how do I extract the "City" value from this json structure and then call a second http_poller with that value as part of my URL?

{
"City" : "Frankfurt",
"Population" : "12345",
"Some stat" : "10"
},
{
"City" : "New York",
"Population" : "123456",
"Some stat" : "100"
}

HTTP poller will run for each city in the "loop":
...
URL => { "https://...../%[current_city_field]" }
...

Thanks in advance !
JD

[magnusbaeck (Magnus Bäck)]

Is it possible in logstash to loop through one of the fields and for each one run a second http_poller?

Not with an http_poller input, but if there were a filter with the same features as http_poller perhaps it would be possible. Right now you'll have to write some Ruby code, either in a ruby filter or in a plugin. If you deserialize the JSON in the HTTP response by using a json codec in the http_poller input you won't have to worry about JSON deserialization in the ruby filter.

A possible option, if you're able to periodically dump all the city mappings into a flat file, is to use a translate filter.