Referencing a weight value from array of match under score function

[Harinder_Singh]

We have a requirement to override default elastic score mechanism and score two documents equally if they have equal number of matches ignoring tf and idf.

Below is my sample index data

PUT my_index/_doc/5
{
  "affinity": {
    "visitorSegments": [
      "TPRO",
      "LPRO"
    ],
    "category": [
      "kitchen",
      "corridor"
    ]
  },
  "sets": [
    {
      "visitorSegments": "TPRO",
      "category" : "kitchen",
      "engagementScore": "0.5"
    },
    {
      "visitorSegments": "LPRO",
      "category" : "kitchen",
      "engagementScore": "0.4"
    },
    {
      "visitorSegments": "LPRO",
      "category" : "corridor",
      "engagementScore": "0.1"
    }
  ],
  "distinctaffinities": 2
}

PUT my_index/_doc/6
{
  "affinity": {
    "visitorSegments": [
      "TPRO"
    ],
    "category": [
      "kitchen"
    ]
  },
  "distinctaffinities": 2
}

Each document is tagged to some visitorSegments and categories. And also an array with possible combinations of values from visitorSegments and category with an enagagementScore.

I am trying the below Query

GET my_index/_search
{
  "query": {
    "bool": {
      "minimum_should_match": 1,
      "should": [
        {
          "function_score": {
            "query": {
              "bool": {
                "minimum_should_match": 1,
                "should": [
                  {
                    "bool": {
                      "must": [
                        {
                          "match_phrase": {
                            "affinity.visitorSegments": "TPRO"
                          }
                        }
                      ]
                    }
                  },
                  {
                    "bool": {
                      "must": [
                        {
                          "match_phrase": {
                            "affinity.category": "kitchen"
                          }
                        }
                      ]
                    }
                  }
                ]
              }
            },
            "functions": [
              {
                "filter": {
                  "match": {
                    "affinity.category": "kitchen"
                  }
                },
                "weight": 3
              },
              {
                "filter": {
                  "match": {
                    "affinity.visitorSegments": "TPRO"
                  }
                },
                "weight": 1
              },
              {
                "filter": {
                  "bool": {
                    "must": [
                      {
                        "match": {
                          "sets.visitorSegments": "TPRO"
                        }
                      },
                      {
                        "match": {
                          "sets.category": "kitchen"
                        }
                      }
                    ]
                  }
                },
                "weight":  //dynamic value for engagementScore for matching entry from sets[]
              }
            ],
            "score_mode": "sum",
            "boost_mode": "replace"
          }
        }
      ]
    }
  }
}

I need to get the value engagementScore from the array entry under sets and set that as value to weight under the third filter statement. So my score should be addition of all three weight statement but third weight value needs to be referred from inside the document only something like
weight : "doc['sets.engagementScore]" and overall expecting score of 3 + 1 + 0.5 = 4.5
Able to get score for 4, but unable to get the additional 0.5

[RabBit_BR (andre.coelho)]

Hi @Harinder_Singh

Try use script_score. I recommend avoid string to field like "engagementScore": "0.1", is better to be double for avoid cast in script.

    {
      "script_score": {
        "script": """ 
        for (item in params._source.sets)
        {
          if(item.visitorSegments == 'TPRO' && item.category == 'kitchen') {
              return Double.parseDouble(item.engagementScore);
          }
        }
        return 0;
        """
      }
    }

Full Query:

{
  "query": {
    "function_score": {
      "query": {
        "bool": {
          "minimum_should_match": 1,
          "should": [
            {
              "bool": {
                "must": [
                  {
                    "match_phrase": {
                      "affinity.visitorSegments": "TPRO"
                    }
                  }
                  ]
              }
            },
            {
              "bool": {
                "must": [
                  {
                    "match_phrase": {
                      "affinity.category": "kitchen"
                    }
                  }
                  ]
              }
            }
            ]
        }
      },
      "functions": [
        {
          "filter": {
            "match": {
              "affinity.category": "kitchen"
            }
          },
          "weight": 3
        },
        {
          "filter": {
            "match": {
              "affinity.visitorSegments": "TPRO"
            }
          },
          "weight": 1
        },
        {
          "script_score": {
            "script": """ 
            def sets = params._source.sets;
            for (item in params._source.sets)
            {
              if(item.visitorSegments == 'TPRO' && item.category == 'kitchen') {
                  return Double.parseDouble(item.engagementScore);
              }
            }
            return 0;
            """
          }
        }
        ],
        "score_mode": "sum",
        "boost_mode": "replace"
    }
  }
}

Response:

  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": 4.5,
    "hits": [
      {
        "_index": "index",
        "_id": "HI-cOYgBkClv0TmBqXNF",
        "_score": 4.5,
        "_source": {
          "affinity": {
            "visitorSegments": [
              "TPRO",
              "LPRO"
            ],
           .....
    ]
  }

[Harinder_Singh]

Hi @RabBit_BR

Thanks for the prompt reply. I will have documents like below as well

PUT my_index/_doc/6

{
  "affinity": {
    "visitorSegments": [
      "TPRO"
    ]
  },
“sets”:[
{
“visitorSegments”: “TPRO”,
“engagementScore”: 0.8
}
]
  "distinctaffinities": 1
}

We will be going for partial match as well , so this document should also be returned with score of 1.8 because it is missing category.But don’t want to add extra conditions in the if, else under script_score.

If we modify the lookup under sets array instead of affinity array, can we still get the engagementScore added to final score per document where each set will be tagged differently.

[RabBit_BR (andre.coelho)]

@Harinder_Singh

If we modify the lookup under sets array instead of affinity array, can we still get the engagementScore added to final score per document where each set will be tagged differently.

does this mean that you got the result you expected without script_score?

@Harinder_Singh

We will be going for partial match as well , so this document should also be returned with score of 1.8 because it is missing category.But don’t want to add extra conditions in the if, else under script_score.

does this mean that you got the result you expected without script_score?

Well... if it is not the condition for adding 0.5, you can change the return 0 to 1.8 (which would be the default). I don't know any other way without adding other conditions in the script_score.