.NET Client: Implementing Location Bias in a Query

Elastic Stack Elasticsearch
1

Hello,
I'm working on a query with location bias with the new Elastic.Clients.Elasticsearch but I can't find any good documentation on how to do it.
My EF-Core entity has those variables:

public double Latitude { get; set; }
public double Longitude { get; set; }

Could anyone give me a tip on how this would need to be indexed or mapped, and how I would construct a location bias within a query?
Right now I get this error: Elastic.Transport.UnexpectedTransportException: The given key 'Coordinates' was not present in the dictionary. with this chunk of code:

.GeoDistance(g => g
      .Field(f => new []{f.Latitude, f.Longitude})
      .Distance($"{radius}")
      .Location(new []{latitude, longitude})
)
2

Hi @Patricius,

could you please share a minimal reproducer including your entity (only the relevant parts), the way you create the index/mappings and the missing parts of your query?

I think this is not really a .NET issue but rather missing knowledge about geo location queries in Elasticsearch in general.

Here is a bit more material:

  1. When using lat/lon coordinates, they must be in a (sub-)object like this:
public sealed record Coordinates
{
    [JsonPropertyName("lat")] // <- important for ES to recognize the coordinate
    public double Latitude { get; init; }
    [JsonPropertyName("lon")] // <- important for ES to recognize the coordinate
    public double Longitude { get; init; }
}

public sealed record MyEntity
{
    // ...

    [JsonPropertyName("coords")]
    public Coordinates Coords { get; init; }
}

Alternatively to defining the custom Coordinates type, the built-in LatLonGeoLocation may be used instead.

  1. When creating the index/mappings, the Coords field must be mapped as geo_point. Mapping is documented here: Custom mapping examples | Elasticsearch .NET Client [8.17] | Elastic

  2. The field in the query must point to the Coordinates/LatLonGeoLocation property:

.GeoDistance(g => g
    .Field(f => f.Coords)
    .Distance($"{radius}")
    .Location(new []{latitude, longitude})
    // alternative: .Location(new LatLonGeoLocation { Lat = latitude, Lon = longitude })
)
3

Thank you for the wonderful response Florian, this is true, I was lacking knowledge and I couldn't find any up-to-date documentation on this.

Thanks to your comment it works for me now.
First I updated my EF-Core entity to include this new record:

public double Latitude { get; set; }
public double Longitude { get; set; }
public Location Location { get; set; }

public record Location(double Lat, double Lon);

This isn't the cleanest solution but it works for testing.

Then I had to map this point to GeoPoint before indexing like so:

await _client.Indices.CreateAsync("myentity", c => c
            .Mappings(m => m
                .Properties<MyEntity>(p => p
                        .GeoPoint(f => f.Location)
                )
            )
);

Now this query works without errors:

var response = await _client.SearchAsync<MyEntity>(s => s
                .Index("myentity")
                .Query(q => q
                        .FunctionScore(fs => fs
                            .Query(qs => qs.Bool(b => b
                                .Should(
                                    bs => bs.GeoDistance(g => g
                                        .Field(f => f.Location)
                                        .Distance($"{radius}m")
                                        .Location(GeoLocation.LatitudeLongitude(new LatLonGeoLocation
                                        {
                                            Lat = latitude,
                                            Lon = longitude
                                        }))
                                    )
                                )
                            ))
                        )
                ),
            cancellationToken
);

I still have some questions regarding my current method:

  • Do I really have to always map this Location variable to GeoPoint? I serialize this new variable Location to json in my database and I want to use Logstash for indexing in production, so this could cause issues later.
    This is how the variable looks like inside of my table column right now: {"lat": 21.34532958, "lon": 9.234154}.
  • My ideal solution would be to not use a record for coordinates, as I don't want EF-Core to create a new table for those entries, and storing them as Json isn't ideal either. Is there a workaround where I can keep using my existing Latitude and Longitude variables?
4

Hi @Patricius , you are welcome :slightly_smiling_face:

From my perspective as a .NET developer I could give you the advice to not store the EF entity in Elasticsearch in the first place.

I would strongly recommend to create a second entity/DTO which you use exclusively for Elasticsearch. AutoMapper might be a solution to map from/to this DTO, if you don't like manual mapping (I personally think manual mapping is the most easy way).

The reasons for this are:

  1. Clean architecture (separation of concerns, etc.)
  2. You don't have to "pollute" your EF entity with the System.Text.Json annotations
  3. Allows you to store the lat/long in the EF entity as regular "flat" properties

If you are rather looking for an Elasticsearch based solution, I'm probably the wrong person to ask (but there are many other experts here in the forums :slightly_smiling_face: ). I know that we have ingest pipelines which can be used to transform ingested data (e.g. to transform the flat lat/long to a proper coordinates field).

1 Like