Upload GeoJSON files using NEST

Elastic Stack Elasticsearch
1

Hi,
We are in the process of moving our search infrastructure from Azure to Elastic Cloud.
We are in a bit of a problem in that we need to know how to upload GeoJsoN files into elastic indexes using the NEST client library preferably. A sample of a line of the fie would look like this:

{ "type": "Feature", "properties": { "id": 1110000065, "localename": "Ashton Vale", "u_loc_name": "Ashton Vale", "town_id": 1120000006, "town_name": "Bristol", "census11pp": 2361, "shoptownid": 2110000082.0, "shoptown": "Bristol" }, "geometry": { "type": "MultiPolygon", "coordinates": [ [ [ [ -2.624719741319515, 51.439378434967367 ], [ -2.626575632279147, 51.439368440970007 ], [ -2.626531740798072, 51.438319355689266 ], [ -2.627193650259407, 51.438119400449821 ], [ -2.627601721439445, 51.437531384459632 ], [ -2.627607711920476, 51.437518407809023 ], [ -2.626421719857078, 51.4358783617729 ], [ -2.624538726794169, 51.436030349054121 ], [ -2.623347745270761, 51.434884260157411 ], [ -2.62258174065521, 51.434745310934758 ], [ -2.621567676055621, 51.43382533380283 ], [ -2.621344571694266, 51.433703273243729 ], [ -2.620585563593697, 51.433562296271454 ], [ -2.619210630089794, 51.432966294962888 ], [ -2.618697564047662, 51.432925315844265 ], [ -2.618656541789477, 51.432935330667675 ], [ -2.619351593062826, 51.435312295595651 ], [ -2.617916581487398, 51.435022322454309 ], [ -2.616802580309487, 51.433941293409816 ], [ -2.616081587073377, 51.434368300530316 ], [ -2.614499502222443, 51.434172277383603 ], [ -2.614024466485181, 51.434500268693682 ], [ -2.613891413139415, 51.434520292480777 ], [ -2.613741512808614, 51.434522335838494 ], [ -2.613127455970896, 51.433858273784779 ], [ -2.612433408874272, 51.43408528551749 ], [ -2.611785409594264, 51.433906276188594 ], [ -2.611686432534913, 51.43389330872597 ], [ -2.611510411169434, 51.433733318691282 ], [ -2.610764444034993, 51.433785300158192 ], [ -2.610366519805594, 51.433644263080993 ], [ -2.610025465794028, 51.433685321476254 ], [ -2.609508345028095, 51.433375273733837 ], [ -2.609317298317929, 51.433119352885058 ], [ -2.608961345331786, 51.432976295028055 ], [ -2.609342341566324, 51.432318186807656 ], [ -2.608990318165541, 51.432024178360003 ], [ -2.609264517030951, 51.431095238187453 ], [ -2.609295481222047, 51.431060198790135 ], [ -2.609471502559053, 51.43105218225466 ], [ -2.610742449268598, 51.431424196935126 ], [ -2.611016449786786, 51.431799240829996 ], [ -2.611537492703756, 51.431826188625756 ], [ -2.612424483260899, 51.431599252971004 ], [ -2.612767415459441, 51.431549188434573 ], [ -2.613175401163998, 51.43126425189935 ], [ -2.612999479146958, 51.430730218860511 ], [ -2.614724554151473, 51.430163219900628 ], [ -2.616109591690619, 51.43001319879405 ], [ -2.616536564914914, 51.42942323355134 ], [ -2.618351539449334, 51.42874019553291 ], [ -2.618789516040517, 51.428274216226242 ], [ -2.618935580441749, 51.428051229540699 ], [ -2.620062737960388, 51.427421235415743 ], [ -2.622790695892857, 51.427519230504778 ], [ -2.623979720090697, 51.427187224941996 ], [ -2.62495573061157, 51.426794226963253 ], [ -2.625537758376238, 51.426375097401866 ], [ -2.626024803413713, 51.425966090743188 ], [ -2.627175824982698, 51.425678123328161 ], [ -2.628116821847654, 51.425391090334358 ], [ -2.628158780913168, 51.425398146842085 ], [ -2.634742958173277, 51.424871144217207 ], [ -2.637589020701783, 51.425471197232795 ], [ -2.640129980622688, 51.425805235791493 ], [ -2.640387018868734, 51.426428229715306 ], [ -2.64061805750684, 51.426828250340868 ], [ -2.641067040320038, 51.427878174779138 ], [ -2.640950035778977, 51.428085209185468 ], [ -2.640645014076153, 51.428620212389745 ], [ -2.639291026519652, 51.431734309058875 ], [ -2.640321970474458, 51.432872301930928 ], [ -2.640480035903146, 51.433132308081547 ], [ -2.639945076183364, 51.436203368134578 ], [ -2.638696887012555, 51.437651429075238 ], [ -2.633411845759151, 51.437105417033159 ], [ -2.632358744285132, 51.43903049609262 ], [ -2.630730861949469, 51.43910048240074 ], [ -2.630062811499199, 51.4392764891778 ], [ -2.629286769024587, 51.440442522949731 ], [ -2.628640634891844, 51.440393491895435 ], [ -2.627523712877389, 51.440538449975804 ], [ -2.626632739729123, 51.441660494011231 ], [ -2.626365720764751, 51.441731497963943 ], [ -2.625149706602658, 51.441190450940653 ], [ -2.62506674656369, 51.440492515595878 ], [ -2.624627677235194, 51.439899495195419 ], [ -2.624719741319515, 51.439378434967367 ] ] ] ] } }

Can anyone help
Thanking you in advance for your assistance

2

in that example you should try to configure your geometry of type geo_shape in the mapping.

3

Hi Alexander
Mant thanks for your response.
We are new to Elastic so the questions are:

1-We wanted to do the mapping with NEST but the only types are GeoPoint and GeoShape but not GeoMultiPolygon
2- Elastic seems to have a multipolygon type as well. Should we ignore that and just map as GeoShape?
3-Can bulk upload the file from .NET or do we have to use curl. If we can, how should the data be formatted?
4-If we can do it with NEST hw do we pass the file contents. Can just pass the contents or do we have to create a seperate POCO for each line.
5-Also in the GeoJSON file, the geometry is a seperate attribute to properties. Do we need to put this inside the properties declaration for mapping.

Can we pay someone just to create a snippet using NEST as an example since there are a few unknowns we don't know what to do with and is delaying our deployment.

Greatly appreciate your assistance
Regards

4

Hey @han1, to answer your questions

Elasticsearch supports geo_point and geo_shape field mappings for geo data. A GeoJSON MultiPolygon would be mapped as a geo_shape type. The geo_shape type can handle the following GeoJSON objects

  • Point
  • LineString
  • Polygon
  • MultiPoint
  • MultiLineString
  • MultiPolygon
  • GeometryCollection

as well as some shapes that are not defined within the GeoJSON spec

  • envelope
  • circle

You could do this using the low level client in Elasticsearch.Net to create bulk requests. You may want to pull out the Feature attributes within the properties into top level properties on the document though and transform perhaps to a friendlier POCO e.g.

public class Location
{
    public int Id { get; set; }
    public string LocaleName { get; set; }
    // ....
    public IGeometry Geometry { get; set; }
}

for working with GeoJSON geometries on your documents with NEST, something like NetTopologySuite is a good choice because your POCO can have an IGeometry property type for the geometry. I don't have an example to hand, but there's some examples of working with NetTopologySuite in this recent issue.

A more comprehensive example is up on my blog but it's a little more out of date, working with NEST 1.x and Elasticsearch 1.x.

5

Hi Russ,
Many thanks for your response.
Do we need to add the converters: GeometryConverter and CoordinateConverter for NEST 5 as you have done in your example?

Is there anyone who can help us with some code as the number of variables that we dont know is too many to keep trial and error with.
Many thanks

6

Yes, if you will be serializing/deserializing types from NetTopologySuite in your application, then you'll need to add the converters using a derived serializer similar to what you see in the issue.

7

Hi Russ
Thanks again.

In NEST 5 there is no .SetJsonSerializerSettingsModifier on the
ConnectionSettings.

Is there any chance to update the example to NEST 5 ? We would be willing
to pay for the trouble.

In addition why can we just not use GeoShape and just dump the GeoJSON
field into that field ?

Regards
Hassan

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.