ES NEST - How to Create an index and Bulk index dynamic objects with geometry (geo_point or geo_shape)?

Elastic Stack Elasticsearch
1

Hi, guys!
I have a Dataset for which I don't know or have the type in advance, nor the number of properties or their type.

On execution, I obtain for that Dataset a DatasetSchema that contain the names, types and some flags for the properties.

For geometry properties I have their GeoJson representation stored as string and I have some flags (isGeoShape, isGeoPoint) that tell the ES property type.

I'm also using NetTopologySuite if it's needed to parse those GeoJsons to actual Geometry objects, but i rather not do this extra parsing and use the GeoJson strings instead.

class DatasetSchema {
	List<DatasetField> Fields;
}

class DatasetField { 
	string Name;
	Type DataType;
	bool isGeoShape;
	bool isGeoPoint;
}

Q:

  1. How can I create such an ES index with unknown / dynamic mappings schema with NEST high level client with those geometry properties?

  2. How can I Bulk index those documents with NEST high level client with Bulk or BulkAll APIs with those geometry properties?

I saw here and here that the bulk indexing might be done with BulkDescriptor:

dynamic obj = new System.Dynamic.ExpandoObject();
	// ….
var descriptor = new BulkDescriptor();
foreach (var doc in values)
{
    descriptor.Index<object>(i => i
        .Index("abc")
        .Id((Id)doc.Id)
        .Document((object)doc));
}
client.Bulk(descriptor);

Still, I’m curious how geometry types should be treated?

Thank you very much!
Any thoughts or suggestions are welcomed!

2

Unfortunately, as you are experiencing, there is no automatic detection for geometric fields.

If you can't create the mapping in advance you could have a dynamic template assuming the field name follows a pattern. So for example you could set a rule for any new field named location or geometry to be mapped automatically as a geo_shape.

Hope this helps!

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.