Scoring script not applied unless I use deprecated filtered query

Elastic Stack Elasticsearch
1

I'm using the Java API (v 2.3.1) to run a custom scoring script as part of a query.

However, I've noticed that the scoring script is not applied unless I use a deprecated part of the API.

ie: This:

BoolQueryBuilder bool = QueryBuilders.boolQuery().minimumNumberShouldMatch(6);
... add bool limiters here via bool.should(...) ...

Map<String, Object> params = new HashMap<>();
... add script parameters here ...

QueryBuilder query = QueryBuilders.functionScoreQuery(bool).setMinScore(12.0f).add(ScoreFunctionBuilders.scriptFunction(new Script("hamming_distance", ScriptService.ScriptType.INLINE, "native", params)))
...execute query here ...

Produces this query json:

{
  "function_score" : {
    "query" : {
      "bool" : {
        "should" : [ ...bool limiters here... ],
        "minimum_should_match" : "6"
      }
    },
    "functions" : [ {
      "script_score" : {
        "script" : {
          "inline" : "hamming_distance",
          "lang" : "native",
          "params" : {
            ...script params here...
          }
        }
      }
    } ],
    "min_score" : 12.0
  }
}

and the scoring script is not applied.

Whereas this:

BoolQueryBuilder bool = QueryBuilders.boolQuery().minimumNumberShouldMatch(6);
... add bool limiters here via bool.should(...) ...

Map<String, Object> params = new HashMap<>();
... add script parameters here ...

QueryBuilder filter = QueryBuilders.filteredQuery(null, bool); // This is deprecated
QueryBuilder query = QueryBuilders.functionScoreQuery(filter).setMinScore(12.0f).add(ScoreFunctionBuilders.scriptFunction(new Script("hamming_distance", ScriptService.ScriptType.INLINE, "native", params)))
...execute query here ...

produces this query json:

{
  "function_score" : {
    "query" : {
      "filtered" : {
        "filter" : {
          "bool" : {
            "should" : [ ...bool limiters here... ],
            "minimum_should_match" : "6"
          }
        }
      }
    },
    "functions" : [ {
      "script_score" : {
        "script" : {
          "inline" : "hamming_distance",
          "lang" : "native",
          "params" : {
            ...script params here...
          }
        }
      }
    } ],
    "min_score" : 12.0
  }
}

and the scoring script is correctly applied.

Am I doing this right?

Thanks for your time!

2

Hey,

have you tried to replace the filtered keyword with bool - that should work as well (sorry not tested, just a hunch after a quick view).

--Alex

3

So I modified the code to replace filter with an outer bool and put the inner bool within a must, like so:

BoolQueryBuilder bool = QueryBuilders.boolQuery().minimumNumberShouldMatch(6);
... add bool limiters here via bool.should(...) ...

Map<String, Object> params = new HashMap<>();
... add script parameters here ...

QueryBuilder filter = QueryBuilders.boolQuery().must(bool);
QueryBuilder query = QueryBuilders.functionScoreQuery(filter).setMinScore(12.0f).add(ScoreFunctionBuilders.scriptFunction(new Script("hamming_distance", ScriptService.ScriptType.INLINE, "native", params)))
...execute query here ...

which produces the following query json:

{
  "function_score" : {
    "query" : {
      "bool" : {
        "must" : {
          "bool" : {
            "should" : [ ...bool limiters here... ],
            "minimum_should_match" : "6"
          }
        }
      }
    },
    "functions" : [ {
      "script_score" : {
        "script" : {
          "inline" : "hamming_distance",
          "lang" : "native",
          "params" : {
            ..script params here...
          }
        }
      }
    } ],
    "min_score" : 12.0
  }
}

Alas, the scoring script is still not applied.

It appears as though an API option has been deprecated and the replacement functionality doesn't work in the same way.

4

Hey,

that was not what I meant, sorry. I meant

{
  "function_score" : {
    "query" : {
      "bool" : {
        "filter" : {
          "bool" : {
            "should" : [ ...bool limiters here... ],
            "minimum_should_match" : "6"
          }
        }

Can you compile a full reproducible example? Including mapping, scripts, documents and query (best in console syntax so one can put it into the kibana console and reproduce easily).

--Alex

5

Ahh ok! Sorry about that.

So I modified it to use bool filter, like so:

 ...
        QueryBuilder filter = QueryBuilders.boolQuery().filter(bool);
...

which produced query json:

{
  "function_score" : {
    "query" : {
      "bool" : {
        "filter" : {
          "bool" : {
            "should" : [ ...bool limiters here... ],
            "minimum_should_match" : "6"
          }
        }
      }
    },
    "functions" : [ {
      "script_score" : {
        "script" : {
          "inline" : "hamming_distance",
          "lang" : "native",
          "params" : {
            ...script params here...
          }
        }
      }
    } ],
    "min_score" : 12.0
  }
}

Interestingly, it looks as though something else is not happening as with that version. It returns 0 hits, rather than the expected 1 hit.

I can try and compile the full test harness if that helps. To start with I just wanted to see if anyone knew what I was doing wrong, because the docs don't make it obvious.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.