Issue using Terms Lookup query inside Percolator

Elastic Stack Elasticsearch
1

Hi! I'm facing a problem using a terms lookup query inside a percolator query.
The actual problem is that the terms are not re-fetched until I reindex the document with the percolator query. I was expecting that, if I update the document referenced by the terms lookup query, and I performed a percolate, the new updated terms will be used.

I'm currently using ES 7.9.

I found a reference of what I think the problem is, under percolator ES documentation:

Fetching queries

There are a number of queries that fetch data via a get call during query parsing. For example the terms query when using terms lookup, template query when using indexed scripts and geo_shape when using pre-indexed shapes. When these queries are indexed by the percolator field type then the get call is executed once. So each time the percolator query evaluates these queries, the fetches terms, shapes etc. as the were upon index time will be used. Important to note is that fetching of terms that these queries do, happens both each time the percolator query gets indexed on both primary and replica shards, so the terms that are actually indexed can be different between shard copies, if the source index changed while indexing.

If this actually is the reason behind my issue, anyone knows if there is a mechanism in order to force the document with the percolator query to re-fetch the terms lookup after the referenced document has been updated? Again, the workaround I found is to update the document manually (with the same body, no change at all, only perform the update just to force terms lookup refresh) but I want to know if there is a better way.

Thank your for your time!
Regards,
Patricio.

2

Hello @patricio.gorin, your conclusion seems right to me. When you use a terms lookup query, then the percolator always uses the terms at the time the percolator query was indexed. Changes to the terms lookup document won't be reflected in the percolator query. The only workaround is what you're already doing -- reindexing the percolator query every time the terms lookup document is changed.

Let us know if you think we could improve anything in the docs that would have helped with your debugging.

Julie

3

Hi Julie!!!

First of all, thank you for your feedback. Even when you confirm that the only way to workaround this issue is to perform an update to the percolator query document, for every update made to the terms lookup referenced document, I need to believe there have to be another way, because under this approach there is no reason at all to use the percolator & terms lookup query combination (I think the usage of terms lookup is just to keep separate a full list of terms and avoid to fetch it every time I need it). I can perfectly persist the lookup terms directly to the percolator query and use a simple terms query, don't you think?

Again, thanks a lot for your help.
Regards,
Pato.

4

Hi @patricio.gorin, this is a good point! That would indeed be a good simplification to the approach.

Julie

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.