Hi,
We investigate the possibility of implementing a new cache mechanism in Elasticsearch.
Here is my use case:
I have some very complex queries that involves many huge full text fields. Those queries can take up to half a minute to be computed. This performance is OK for our business.
However, we want to be able to cache this query efficiently.
It is not common that the user wants to reuse the query. The shard query cache is ok for that.
Another use case for the user is to combine two previous queries with simple operators (let's say : 1 AND 2)
The shard query cache won't be helpful for that as it will consider that 1 AND 2 is a new query.
Adding a specific parsing plugin for that could be a solution (I assume that each filter cache entry is an ordered list of docid, that can easily be merged with simple boolean rules). The problem with that approach is that we loose scoring, but I think we have no other choice if you want to avoid reexecuting the whole query.
Have someone ever tried this kind of approach?
Regards