Join data from an ES index with data from logstash

(Tolivier) #1

I have indexed data from a referential (a list of providers, with their full name) from a file.
With logstash, I index data from a DB that contains also the ID of providers.

I want, when reading data from DB with logstash, populate the full name of the provider, in order to have an index with denormalized data.

How can I do this ?

I worked with another search engine and I have realized a customized plugin, reading data from an index and storing then in memory, so, on the fly, I could add data during the load in the index.

(Mark Walkom) #2

If the data is in ES then you can use the ES filter to do something like this.
Otherwise look at the translate plugin.

(system) #3