Script result caching/reusing

Elastic Stack Elasticsearch
1

Hi there,

I have a similar use-case as in https://github.com/elastic/elasticsearch/issues/10767

  • price calculation data is stored in a children index (I know that this makes things more complex)
  • one of the aggregations is based on price e.g. multiple price ranges 50 <= price < 100, 100 <= price < 150 etc
  • sorting can use such price
  • post_filter can filter on such price
  • a top hits aggregation of the price children objects is always needed, and it uses such price calculation; although I am doing this through a separate regular ES query at the moment, I plan to use top hits aggregation in a single query as explained here

The price calculation has a dependency on user request parameters, hence I use scripts (Lucene expressions).

Since there is no script result caching (as far as I understood), I am trying to re-engineer this around usage of function score. A few questions:

  • can you foresee a better way to accomplish this?
  • can function score be cached for re-use across queries?
  • do I need to copy the post_filter as a filter in the top hits aggregation? (to avoid calculating top hits for results that will be skipped)

Thanks!

1 Like
2

I had almost the same questions recently. I don't know where to find any help with such use-case, but at least I understood there are no solutions in ES at the moment to optimize requests for the large index with many documents, especially if you have many aggs, filters and sort. I'd say the best thing could be done is to write to the github issues: https://github.com/elastic/elasticsearch/issues/10767 and https://github.com/elastic/elasticsearch/issues/20131

My question is also uncommented: Generate multiple scripted fields for the whole index to filter and aggregate

Looks like here is nobody help anyone, at least for the complex questions, where the documentation cannot help.

3

I think this 'discuss' does not work really well..at least for complex questions. We might actually have better luck at StackOverflow I think.

We are also non-paying customers, that doesn't help :slight_smile:

1 Like
4

I think I've already asked this question on StackOverflow. I believe the only help we can wait for is from the the ES developers.
I had many problems with ES for the last years so far, and always everything should be solved by myself. Nobody can help, as there are not so many developers who are working with complex queries in ES. Mostly simple questions could be solved, like how to setup replication, discovery of nodes, how to work with text queries. But everything beyond this is troublesome.