Scoring by distance

Elastic Stack Elasticsearch
1

Hi,

I need to set score by distance and I found good example - https://www.elastic.co/guide/en/elasticsearch/guide/current/decay-functions.html - but it looks not working in my case.

My request:

GET earth/destination/_search?_source=country_name,city_name,location&size=3
{
  "query": {
    "function_score": {
      "query": {
        "match_all": {}
      },
      "functions": [
        {
          "gauss": {
            "location": {
              "origin": { "lat": 0, "lon": 50 },
              "scale": "10km"
            }
          }
        }
      ]
    }
  }
}

My response:

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 4,
    "successful": 4,
    "failed": 0
  },
  "hits": {
    "total": 572,
    "max_score": 0,
    "hits": [
      {
        "_index": "earth",
        "_type": "destination",
        "_id": "450",
        "_score": 0,
        "_source": {
          "city_name": "Piran",
          "country_name": "Slovenia",
          "location": {
            "lon": "13.5682895",
            "lat": "45.528319"
          }
        }
      },
      {
        "_index": "earth",
        "_type": "destination",
        "_id": "452",
        "_score": 0,
        "_source": {
          "city_name": "Dubai",
          "country_name": "United Arab Emirates",
          "location": {
            "lon": "55.307485",
            "lat": "25.271139"
          }
        }
      },
      {
        "_index": "earth",
        "_type": "destination",
        "_id": "462",
        "_score": 0,
        "_source": {
          "city_name": "Villefranche-Sur-Mer",
          "country_name": "France",
          "location": {
            "lon": "7.311109",
            "lat": "43.703976"
          }
        }
      }
    ]
  }
}

So, all scores are 0. What's wrong in my request?

2

I see scores with "scale": "1000km", but values are very strange

Sorting by distance as described in https://www.elastic.co/guide/en/elasticsearch/guide/current/sorting-by-distance.html works better, but scores are null.

I need right score value, because I have other sort options except destination and 'offset' option of decay function from https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#function-decay is really useful for me too.

3
  1. Are you sure your location mapping is geo point ?
  2. use "origin": latitude + "," + longitude this is the same as lat lon i guess.
  3. you can use weight in gauss scoring function.
4

I tried to use custom score function as described in https://github.com/elastic/elasticsearch/issues/607, but distance function call was failed with:

         "reason": {
          "type": "script_exception",
          "reason": "runtime error",
          "caused_by": {
            "type": "illegal_argument_exception",
            "reason": "Unable to find dynamic method [distance] with [2] arguments for class [org.elasticsearch.index.fielddata.ScriptDocValues.GeoPoints]."
          },
          "script_stack": [
            "1/doc['location'].distance(0, 50)",
            "                 ^---- HERE"
          ],
          "script": "1/doc['location'].distance(0, 50)",
          "lang": "painless"
        }

I've found arcDistance function in https://github.com/elastic/elasticsearch/blob/master/core/src/main/java/org/elasticsearch/index/fielddata/ScriptDocValues.java and tried to use it but result was the same as "gauss" function call with "scale": "1000km", compare:

      {
        "_index": "wg_data",
        "_type": "destination",
        "_id": "1003",
        "_score": 7.241588e-7,
        "_source": {
          "city_name": "Zanzibar",
          "country_name": "Tanzania",
          "location": {
            "lon": "39.19793",
            "lat": "-6.16394"
          }
        }
      }

and:

      {
        "_index": "wg_data",
        "_type": "destination",
        "_id": "695",
        "_score": 1.7508285e-7,
        "_source": {
          "city_name": "Dubrovnik",
          "country_name": "Croatia",
          "location": {
            "lon": "18.09479",
            "lat": "42.65372"
          }
        }
      }

The question is the same: what is wring in my request or data?

5

yes, my mapping is:
"location": {
"type": "geo_point"
}

  • is not possble for numbers, result is:
    "type": "json_parse_exception",
    "reason": "Unexpected character ('0' (code 48)) in numeric value: expected digit (0-9) to follow minus sign, for valid numeric value\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@37655b2a; line: 11, column: 36]"
  • is not changing result

How to apply weight to 0 score? Can you show me example?

6

Looks like arcDistance function works wrong. I tried to see distance in response with:

GET wg_data/destination/_search?_source=country_name,city_name,location,distance
{
  "query": {
    "match_all": {}
  },
  "script_fields": {
    "distance": {
      "script": "doc['location'].arcDistance(0,50)"
    }
  }, 
  "sort": [
    {
      "_geo_distance": {
        "location": {
          "lat": 50,
          "lon": 0
        },
        "order": "asc"
      }
    }
  ]
}

results was wrong:

      {
        "_index": "wg_data",
        "_type": "destination",
        "_id": "4053",
        "_score": null,
        "_source": {
          "city_name": "Deauville",
          "country_name": "France",
          "location": {
            "lon": "0.074075",
            "lat": "49.357315"
          }
        },
        "fields": {
          "distance": [
            7250861.542755543
          ]
        },
        "sort": [
          71661.87714205771
        ]
      }

Is this a bug or distance calculation must works in another way?

7

doc['location'].planeDistance(0, 50)

distance is deprecated i guess.

I think function_score is messing with your score.
use boost_mode="replace",
score_mode="sum", in function score for exact score of gauss function
in gauss function
"gauss": {
"location": {
"origin": { "lat": 0, "lon": 50 },
"scale": "10km"
},
"weight":100
}

8

arcdistance will give in meters and sort will give in KM i guess.

9

Sorting sequence with meters and kilimeters must be the same. Are you have another opinion?

10

planeDistance returns the same results.

weight, boost_mode and score_mode can't change score 0

11

when are doing sort, "script_fields" unnecessary computation.
we can get the distance value from sort.

12

"script_fields" is example of wrong distance calculation by arcDistance/planeDistance

1 Like
13

New issue is submitted - https://github.com/elastic/elasticsearch/issues/23178

14

Your location is just too far from your query point, so the decay function results in a very low number.

Try this instead:

GET _search?explain
{
  "query": {
    "function_score": {
      "query": {
        "match_all": {}
      },
      "boost_mode": "replace", 
      "functions": [
        {
          "gauss": {
            "location": {
              "origin": { "lat": 45, "lon": 13 },
              "scale": "100km"
            }
          }
        }
      ]
    }
  }
}
15

Sorry, I simple swapped "lat" and "lon" params and this was a result of strange scores and distances

16

planeDistance is faster to compute, so on sorting planeDistance is used.

Plane Distance will not give exact distance. unlike arcDistance.

but to keep things constant across sorting and script_fields using plane Distance is the best way.

17

Sorry, I simple swapped "lat" and "lon" params and this was a result of strange scores and distances

1 Like
18

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.