Is it possible to do multiple Elasticsearch lookup filters to search multiple indices in a single Logstash pipeline?
Yes, it is.
@Badger can this be done with one elasticsearch filter or two elasticsearch filters? I'm currently using two but it's going through the first filter still, but I'm not seeing the data being added from the other filter/index.
An elasticsearch filter runs a query against a list of indexes. If you can construct a single query that works on both indexes to fetch the data you want then I think it may be possible to do with one filter. However, the configuration may be easier to understand if you use two.
@Badger I tried specifying multiple indices by doing index => ["index1", "index2"] in the elasticsearch filter and the field that I query is found in both indices, so theoretically I'm just adding the fields that needs to be appended. I test the logstash config but it fails when I try to use a single elasticsearch filter, but doesn't when I use two. The only issue is that when I use two filters I don't see the second lookup to the second index. I know this would be alot easier with the enrich processor, but unfortunately I'm using 7.4.
What does that mean?
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.