How to store geoshape Linestring with C#?

Using the latest .NET client Elastic.Clients.Elasticsearch 8.14.3, how can I in code add a geoJSON geoshape of type LineString to Elasticsearch?

The documentation is completely lacking information on this and I have yet to find a working example as I apparently can't figure it out by myself.

For example, how do I add this object to Elasticsearch so that I can later search for matching paths within a bounding box?

private readonly Path _path = new()
{
    Name = "An example path",
    Geometry = new Geometry
    {
        Type = "LineString",
        Coordinates = new[,] {
                    {15.288221000000002, 59.895211999999987},
                {15.287267000000002, 59.895158000000009},
    }
};

@Motsols Hi!

What exactly is the problem here?

The structure of your Geometry class seems to match the LineString format. Inserting this type should work as is.

Hi Florian,

When I try it out I'm getting the below error.

 Exception: Request failed to execute. Call: Status code 400 from: PUT /paths/_doc/1. ServerError: Type: illegal_argument_exception Reason: "mapper [geometry.coordinates] cannot be changed from type [dense_vector] to [float]"

# Audit trail of this API call:
 - [1] BadResponse: Node: http://localhost:9200/ Took: 00:00:00.1223880

# OriginalException: Elastic.Transport.TransportException: Request failed to execute. Call: Status code 400 from: PUT /paths/_doc/1. ServerError: Type: illegal_argument_exception Reason: "mapper [geometry.coordinates] cannot be changed from type [dense_vector] to [float]"

# Request:
<Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on TransportConfiguration to force it to be set on the response.>

# Response:
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"mapper [geometry.coordinates] cannot be changed from type [dense_vector] to [float]"}],"type":"illegal_argument_exception","reason":"mapper [geometry.coordinates] cannot be changed from type [dense_vector] to [float]"},"status":400}

Here's a full code example to try it out with

public class Path
{
    public required string Name { get; set; }
    public required Geometry Geometry { get; set; }
}

public class Geometry
{
    public required string Type { get; set; }

    [JsonConverter(typeof(Array2DConverter))]
    public required double[,] Coordinates { get; set; }
}

You can put this into Program.cs to have it all in one place for testing purposes

var settings = new ElasticsearchClientSettings(new Uri("http://localhost:9200"))
    .Authentication(new BasicAuthentication("elastic", "myPassword"));

var client = new ElasticsearchClient(settings);

var createIndexResponse = await client.Indices.CreateAsync("paths", i => i
    .Mappings(m => m
        .Properties(new(){
            { "Name", new TextProperty() },
            { "Geometry", new GeoShapeProperty() }
        })
    )
);

Path _path = new()
{
    Name = "An example path",
    Geometry = new Geometry
    {
        Type = "LineString",
        Coordinates = new[,] {
                    {15.288221000000002, 59.895211999999987},
                {15.287267000000002, 59.895158000000009},
    }
};

// This is where the error happens
var indexingResult = await client.IndexAsync<Path>(_path, (IndexName)"paths");

This is taken from SO as I didn't get an answer on how to do it on this Elastic forum

using System.Reflection;
using System.Text.Json;
using System.Text.Json.Serialization;

namespace Core.Converters
{
    // This is taken from here https://stackoverflow.com/questions/66280645/how-can-i-serialize-a-double-2d-array-to-json-using-system-text-json 

    public class Array2DConverter : JsonConverterFactory
    {
        public override bool CanConvert(Type typeToConvert) => typeToConvert.IsArray && typeToConvert.GetArrayRank() == 2;

        public override JsonConverter CreateConverter(Type type, JsonSerializerOptions options) =>
            (JsonConverter)Activator.CreateInstance(
                typeof(Array2DConverterInner<>).MakeGenericType([type.GetElementType()]),
                BindingFlags.Instance | BindingFlags.Public,
                binder: null,
                args: [options],
                culture: null);

        class Array2DConverterInner<T> : JsonConverter<T[,]>
        {
            readonly JsonConverter<T> _valueConverter;

            public Array2DConverterInner(JsonSerializerOptions options) =>
                _valueConverter = typeof(T) == typeof(object) ? null : (JsonConverter<T>)options.GetConverter(typeof(T)); // Encountered a bug using the builtin ObjectConverter 

            public override void Write(Utf8JsonWriter writer, T[,] array, JsonSerializerOptions options)
            {
                // Adapted from this answer https://stackoverflow.com/a/25995025/3744182
                // By https://stackoverflow.com/users/3258160/pedro
                // To https://stackoverflow.com/questions/21986909/convert-multidimensional-array-to-jagged-array-in-c-sharp
                var rowsFirstIndex = array.GetLowerBound(0);
                var rowsLastIndex = array.GetUpperBound(0);
                var columnsFirstIndex = array.GetLowerBound(1);
                var columnsLastIndex = array.GetUpperBound(1);

                writer.WriteStartArray();
                for (var i = rowsFirstIndex; i <= rowsLastIndex; i++)
                {
                    writer.WriteStartArray();
                    for (var j = columnsFirstIndex; j <= columnsLastIndex; j++)
                        _valueConverter.WriteOrSerialize(writer, array[i, j], options);
                    writer.WriteEndArray();
                }
                writer.WriteEndArray();
            }

            public override T[,] Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) =>
                JsonSerializer.Deserialize<List<List<T>>>(ref reader, options)?.To2D();
        }
    }

    public static partial class ArrayExtensions
    {
        public static T[,] To2D<T>(this List<List<T>> source)
        {
            // Adapted from this answer https://stackoverflow.com/a/26291720/3744182
            // By https://stackoverflow.com/users/3909293/diligent-key-presser
            // To https://stackoverflow.com/questions/26291609/converting-jagged-array-to-2d-array-c-sharp
            var firstDim = source.Count;
            var secondDim = source.Select(row => row.Count).FirstOrDefault();
            if (!source.All(row => row.Count == secondDim))
                throw new InvalidOperationException();
            var result = new T[firstDim, secondDim];
            for (var i = 0; i < firstDim; i++)
                for (int j = 0, count = source[i].Count; j < count; j++)
                    result[i, j] = source[i][j];
            return result;
        }
    }

    public static class JsonSerializerExtensions
    {
        public static void WriteOrSerialize<T>(this JsonConverter<T> converter, Utf8JsonWriter writer, T value, JsonSerializerOptions options)
        {
            if (converter != null)
                converter.Write(writer, value, options);
            else
                JsonSerializer.Serialize(writer, value, typeof(T), options);
        }
    }
}

I sadly can not reproduce the issue using your code:

Successful (201) low level call on POST: /paths/_doc?pretty=true&error_trace=true

# Audit trail of this API call:
 - [1] HealthyResponse: Node: [...] Took: 00:00:00.1917234
# Request:
{
  "name": "An example path",
  "geometry": {
    "type": "LineString",
    "coordinates": [
      [15.288221000000002,59.895211999999987
      ],
      [15.287267000000002,59.895158000000009
      ]
    ]
  }
}
# Response:
{
  "_index" : "paths",
  "_id" : "sDN2SpAB5uu8gcfdc2g_",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}

Are you sure the document with ID=1 does not already exist in the index? For me it looks like it's already indexed (and created with a different mapping).

I've been fiddling around with this on my spare time and have figured out why my code fails while the smaller LineString example I posted here does work.

The mapping I create specifies that the Geometry is a GeoShapeProperty.
If the first document I add to the new index has a LineString of 98 geo points it fails the seeding citing the error with dense_vector.
If I delete some points to get it down to 43 then it successfully adds it to the index after which I can freely add a LineString with 200 geo points without issue.

The breakpoint seems to be around 50 geo points, above which it decides that it is a dense_vector instead and fails the creation of the document.

To me this seems like a bug.
Is the function as intended or should I report it as a bug in github? If not, am I using it wrong and how should I use it instead?

@Motsols Thanks for your analysis of the problem. For me this sounds like a bug as well. Is that issue reproducible using e.g. CURL or Kibana? I would like to rule out the client as the potential cause for this strange behavior. If it can be reproduced with a different client, you should probably open a GitHub issue in the ES server repository.

I tried the same in Kibana and confirmed the error. Then I went to my previous index to check the index and noticed that it now looks very different compared to when I created the index and the mapping.

{
  "mappings": {
    "properties": {
      "Geometry": {
        "type": "geo_shape"
      },
      "Name": {
        "type": "text"
      },
      "Parent": {
        "type": "text"
      },
      "Type": {
        "type": "text"
      },
      "geometry": {
        "properties": {
          "coordinates": {
            "type": "float"
          },
          "type": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      },
      "id": {
        "type": "long"
      },
      "name": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "parent": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "type": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      }
    }
  }
}

And my code to create the mappings look like this:

var createIndexResponse = await _esClient.Indices.CreateAsync(PATHS_INDEX, i => i
    .Mappings(m => m
        .Properties(new(){
            { "Type", new TextProperty() },
            { "Name", new TextProperty() },
            { "Parent", new TextProperty() },
            { "Geometry", new GeoShapeProperty() }
        })
    )
);

In other words this was a naming issue where I'm writing the index mapping property names with camel case but the serialized json is in lower case, causing the mismatch.

I changed the mapping code to be in lowercase as well and that solved the issue at hand. I can now upload very big LineString documents without issue.

Thanks for your help!

Hi @Motsols,

oh yes, that explains it :slightly_smiling_face: I should have noticed the incorrect naming as well :confused:

The safest way to avoid these issues, is to use the generic overloads with lambda expressions whenever possible:

var createIndexResponse = await client.Indices.CreateAsync<Path>("paths", i => i
	.Mappings(m => m
		.Properties(p => p
			.Text(x => x.Name)
			.GeoShape(x => x.Geometry)
		)
	)
);

This ensures correct casing and as well makes sure your names stay in sync with the POCO type.

Thanks for the suggestion!
I changed my code to use that style. I didn't know it was an option.

Maybe it's worth add an example mapping on this page of the .net documentation? CRUD usage examples | Elasticsearch .NET Client [8.9] | Elastic