Define score value for searches and sum the total

[Reemface (Remi Romme)]

Hi All!

I am fairly new in the world of ElasticSearch, and using it in a very basic way.
I would like to use Elastic in a more complex way, but i can't get it to work in the way i would like.

At this moment we have our own MySQL search engine which is slow...
It is used to search trough products and give a score for every kind of match that is found.

• match in title is 1 points,
• match in brandname is an extra 2 points,
• match in product-category name is 1 extra point,
• etc.

I would like to create this method with Elastic, but i can not get it to use a fixed score for multiple fields.

I did some experiments with BOOL queries with a constant_score. When i searched one field the score did match the 'boost' value, but when searched multiple fields, the score result wasn't a rounded number. For example, i want every product with "cooler" in the name, and give it an extra score when "cooler" is also in the brand name:

{
	"query": {
		"bool": {
			"should": [
			{
				"constant_score": {
					"boost": 1,
					filter: {
						"wildcard": {
							"naam": "*cooler*"
						}
					}
				}
			},
			{
				"constant_score": {
					"boost": 2,
					filter: {
						"wildcard": {
							"merk": "*cooler*"
						}
					}
				}
			}
			]
		}
	},
    "size": 100
}

This results in scores like 1.3416407, where i expect a 3.0 score:

{
    "took": 47,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 599,
        "max_score": 1.3416407,
        "hits": [
            {
                "_index": "products",
                "_type": "product",
                "_id": "153136",
                "_score": 1.3416407,
                "_source": {
                    "artikelnr": 153136,
                    "naam": "Cooler Master Jetflo blauwe led, 120mm",
                    "prodgroep": 10,
                    "subgroep": 11,
                    "groepnaam": "Koeling (lucht)",
                    "subgroepnaam": "Case fan 120mm",
                    "merk": "Cooler Master",
                    "vendor": "R4-JFDP-20PB-R1",
                    "ean": "4719512042267",
                    "synoniem": [
                        ""
                    ],
                    "synced": "2018-10-10 09:48:01"
                }
            },
            {
                "_index": "products",
                "_type": "product",
                "_id": "995569",
                "_score": 1.3416407,
                "_source": {
                    "artikelnr": 995569,
                    "naam": "Cooler Master MasterLiquid Pro 280",
                    "prodgroep": 42,
                    "subgroep": 8,
                    "groepnaam": "Koeling (water)",
                    "subgroepnaam": "Complete sets CPU",
                    "merk": "Cooler Master",
                    "vendor": "MLY-D28M-A22MB-R1",
                    "ean": "4719512052631",
                    "synoniem": [
                        "Waterkoeling"
                    ],
                    "synced": "2018-10-29 01:37:01"
                }
            },
            {
                "_index": "products",
                "_type": "product",
                "_id": "975619",
                "_score": 1.3416407,
                "_source": {
                    "artikelnr": 975619,
                    "naam": "Cooler Master MasterKeys Pro M (White LED) - Brown Switch",
                    "prodgroep": 15,
                    "subgroep": 1,
                    "groepnaam": "Invoerapparaten",
                    "subgroepnaam": "Toetsenborden",
                    "merk": "Cooler Master",
                    "vendor": "SGK-4080-KKCM1-US",
                    "ean": "4719512052129",
                    "synoniem": [
                        ""
                    ],
                    "synced": "2018-10-30 10:19:01"
                }
            },

--- code limited ---

Is there a way to use the scoring mechanism the way i want?

[Reemface (Remi Romme)]

i did get it to work!

{
	"sort": [
		"_score",
		{
			"clicks": "desc"
		}
	],
	"query": {
		"bool": {
			"should": [
				{
					"constant_score": {
						"boost": 1,
						"filter": {
							"term": {
								"naam": "apple"
							}
						}
					}
				},
				{
					"constant_score": {
						"boost": 1,
						"filter": {
							"term": {
								"merk": "apple"
							}
						}
					}
				}
			]
		}
		
	},
	"size": 100
}

This code outputs the documents, with a score of
0 = no mathces
1 = match on brandname of productname
2 = match on brandname AND productname

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 41,
      "relation": "eq"
    },
    "max_score": null,
    "hits": [
      {
        "_index": "megekko",
        "_type": "products",
        "_id": "148599",
        "_score": 2.0,
        "_source": {
          "artikelnr": 148599,
          "naam": "Apple MagSafe / MagSafe 2",
          "prodgroep": 22,
          "subgroep": 11,
          "groepnaam": "Kabels en Connectors",
          "subgroepnaam": "Koppelstukjes",
          "merk": "Apple",
          "vendor": "MD504ZM/A",
          "ean": "885909604203",
          "synoniem": [
            ""
          ],
          "synced": "2019-10-21 13:12:01",
          "clicks": 91
        },
        "sort": [
          2.0,
          91
        ]
      },
      {
        "_index": "megekko",
        "_type": "products",
        "_id": "148591",
        "_score": 2.0,
        "_source": {
          "artikelnr": 148591,
          "naam": "Apple Lightning-naar-USB-kabel 0,5 meter",
          "prodgroep": 22,
          "subgroep": 28,
          "groepnaam": "Kabels en Connectors",
          "subgroepnaam": "Oplaad kabels",
          "merk": "Apple",
          "vendor": "ME291ZM/A",
          "ean": "0885909707973",
          "synoniem": [
            "Lightning kabels",
            "lightningkabels",
            "oplaadkabels"
          ],
          "synced": "2019-10-21 13:12:01",
          "clicks": 81
        },
        "sort": [
          2.0,
          81
        ]
      }
    ]
  }
}

But now there is an additional question:
I would like to search in a group of fields, and only score 1 when the searched value is in one of the fields. I've seen that it is possible to create another bool-query inside the 'should' part, but i can not get it to only score 1 when multiple matches are found.

I would like to have the second 'should' group to score a maximum of 1, even when the searched term is found in all of the fields.

{
	"sort": [
		"_score",
		{
			"clicks": "desc"
		}
	],
	"query": {
		"bool": {
			"should": [{
					"bool": {
							"should": [{
								"constant_score": {
									"boost": 1,
									"filter": {
										"term": {
											"synoniem": "apple"
										}
									}
								}
							},
							{
								"constant_score": {
									"boost": 1,
									"filter": {
										"term": {
											"groepnaam": "apple"
										}
									}
								}
							},
							{
								"constant_score": {
									"boost": 1,
									"filter": {
										"term": {
											"subgroepnaam": "apple"
										}
									}
								}
							}
						]
					}
				},
				{
					"constant_score": {
						"boost": 1,
						"filter": {
							"term": {
								"naam": "apple"
							}
						}
					}
				},
				{
					"constant_score": {
						"boost": 1,
						"filter": {
							"term": {
								"merk": "apple"
							}
						}
					}
				}
			]
		}
	},
	"size": 100
}

[Glen_Smith]

Within your constant_score.filter object, you can have a bool query, so it's easy to have a should in one of them that specifies all the fields you want to "switch on" that boost.

Something like

                "constant_score": {
                  "boost": 1,
                  "filter": {
                    "bool": {
                      "should": [
                        {
                          "term": {
                            "synoniem": "apple"
                          }
                        },
                        {
                          "term": {
                            "groepnaam": "apple"
                          }
                        },
                        {
                          "term": {
                            "subgroepnaam": "apple"
                          }
                        }
                      ]
                    }
                  }
                }

However, I'd like to note that it's better to use match queries with text fields. This way the search token is analyzed in the same way the indexed data was.

Then my snippet becomes

          "constant_score": {
            "boost": 1,
            "filter": {
              "multi_match": {
                "query": "apple",
                "fields": ["synoniem", "groepnaam", "subgroepnaam"]
              }
            }
          }

(The reduced indentation reflects discarding a superfluous containing bool query.)

Another useful feature to be aware of when customizing scoring is Function score query.