Filtering documents by score via scripting

Elastic Stack Elasticsearch
1

Hello,

I have a specific scenario where I have an entity A that has collection of
nested entities of B.

While searching on A index, I need to compute B's that match a certain
criteria and then use scripting to calculate a specific value with specific
params. This value is obtained from some computed information of N elements
of B (sometimes just one B but it's not the rule). Ultimately this value
might (and not necessarily) need to be filtered and/or sorted by.

Initially I started using filter script, loading data through _source, and
instantly I realised that when the dataset is around 10k documents (which
from what I've seen in this forum is very small) the performance degrades
from 300ms (dataset filtered to 2-3k documents) to 2seconds in one single
request. A major problem also if the cluster receives a lot of these slow
queries, it will degrade the overall response time of everything. (I know
at this point RAM is enough and I'm already using a SSD).

I started looking for options and realised function_score could be a good
candidate to avoid the nasty _source. I managed to come up with a way to
calculate the value I need - still not very fast though - 800ms for the
worst case scenario ( maybe can live with this, if 95% of my other queries
are < 100ms).

  1. Should I be using function_score for something that I need filtering
    or sorting ?
  2. Regarding filtering the score ES allows a min_score, but not a
    max_score. The following query fails with this
    error: GroovyScriptExecutionException[GroovyRuntimeException[Cannot compare
    org.elasticsearch.script.ScoreAccessor with value
    'org.elasticsearch.script.ScoreAccessor@7c792ee' and java.lang.Integer with
    value '300'. Looking at the source code, do I need to call doubleValue() ?
    The query does start to work but I wonder why this is not in the
    documentation ?

{
"min_score": 0,
"query": {
"function_score": {
"boost_mode": "multiply",
"query": {
"nested": {
"path": "nested_path",
"query": {
"function_score" : {
"score_mode": "sum",
"script_score": {
"script": "...",
"params": {
...
}
}
}
}
}
},
"script_score" : {
"params" : { "min": 300, "max": 500 },
"script" : "(_score > min) && (_score < max) ? 1 : -1"
}
}
}
}

Thanks in advance for any tips in this scenario, I would greatly appreciate
any help & suggestions,
Pedro

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/aa1036c2-05e8-4ba6-b493-0a90951e5058%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.