Use a logstash plugin within iteration

Elastic Stack Logstash
1

Hello,
I've got documents where a field might contain multiple values.
All the values from that field have to be looked up in a different index, to get the document enriched.

"countries" : "India|Congo|Italy"

I should iterate over the n values and invoke the elastic plugin for each value:

      mutate {
         split => { "countries" => "|" }
      }
      
      ## sort of foreach country in countries
      elasticsearch {
         hosts => ["es-server"]
         index => "geo_data"
         query_template => "countryToContinent.json"
         fields => { 
            "Continent" => "continents[]"
            "OfficialCountryName" => "country_official_names[]"
         }
      }

countryToContinent.json:

    {
      "size": 1,
      "_source": ["Continent", "OfficialCountryName"],
      "query": {
        "query_string": {
          "query":  ***  "%{countries[i]}"  ***
        }
      }
    }

I'm missing how to iterate over countries array and invoke the elasticsearch plugin (but an example with any other plugin would be useful) for each value.
Additionally, also continents and country_official_names should end up being array fields.

I've seen How to loop through array in Logstash post and solution, but it doesn't really help me.

Any other suggestion to tackle the problem is welcome.

1 Like
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.