Nest elasticsearch 7 Geolocation indexing

Please help me with indexing geolocation in nest elastic search 7, this is my code
Model

public class ElasticSearchModel : Model<Building>
{     
        public ElasticSearchModel()
        {
            BuildingRooms = new List<BuildingRoomViewModel>();
            BuildingPhotos = new List<BuildingPhotoViewModel>();
        }

        public GeoLocation Location { get; set;}
        public int Id { get; set; }
        public string BuildTitle { get; set; }
        public DateTime CreatedDate { get; set; }
        public int TotalRooms { get; set; }
        public string Description { get; set; }
        public decimal? OwnerPrice { get; set; }
        public double? Size { get; set; }
        public string SizeName { get; set; }
        public string MoneyType { get; set; }
        public string BuildAction { get; set; }       
}

and creating index like this

 _elasticClient.CreateIndex("honadona", c => c
                            .Map<ElasticSearchModel>(mm => mm
                                .AutoMap()
                            
                        )
                    );

but i get response structure like this for location field:

"location": {
  "properties": {
    "lon": {
      "type": "float"
    },
    "lat": {
      "type": "float"
    }
}

I cant search with Geodistance, it give me error "all_shard":

var Buildingpoints= _elasticClient.Search<ElasticSearchModel>(
                     s => s.RequestConfiguration(r => r
        .DisableDirectStreaming() 
    ).From(0).Size(600).Query(query => query.Bool(b => b
                     .Filter(filter => filter
                    .GeoDistance(geo => geo
                   .Field(f => f.Location)
                   .Distance(4.0, Nest.DistanceUnit.Kilometers)
                   .Location(41.23, 69.32)
                    )

Please format your code using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:

```
CODE
```

This is the icon to use if you are not using markdown format:


It looks like you're using NEST 7.x, based on the create index API call. I can't replicate what you're seeing with NEST 7.0.0-alpha2

private static void Main()
{
    var defaultIndex = "example_index";
    var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));

    var settings = new ConnectionSettings(pool)
        .DefaultIndex(defaultIndex);

    var client = new ElasticClient(settings);

    var createIndexResponse = client.CreateIndex("honadona", c => c
        .Map<ElasticSearchModel>(mm => mm
            .AutoMap()
        )
    );
}

public class ElasticSearchModel
{
	public GeoLocation Location { get; set; }
	public int Id { get; set; }
	public string BuildTitle { get; set; }
	public DateTime CreatedDate { get; set; }
	public int TotalRooms { get; set; }
	public string Description { get; set; }
	public decimal? OwnerPrice { get; set; }
	public double? Size { get; set; }
	public string SizeName { get; set; }
	public string MoneyType { get; set; }
	public string BuildAction { get; set; }
}

produces the following request

PUT http://localhost:9200/honadona?pretty=true 
{
  "mappings": {
    "properties": {
      "location": {
        "type": "geo_point"
      },
      "id": {
        "type": "integer"
      },
      "buildTitle": {
        "fields": {
          "keyword": {
            "ignore_above": 256,
            "type": "keyword"
          }
        },
        "type": "text"
      },
      "createdDate": {
        "type": "date"
      },
      "totalRooms": {
        "type": "integer"
      },
      "description": {
        "fields": {
          "keyword": {
            "ignore_above": 256,
            "type": "keyword"
          }
        },
        "type": "text"
      },
      "ownerPrice": {
        "type": "double"
      },
      "size": {
        "type": "double"
      },
      "sizeName": {
        "fields": {
          "keyword": {
            "ignore_above": 256,
            "type": "keyword"
          }
        },
        "type": "text"
      },
      "moneyType": {
        "fields": {
          "keyword": {
            "ignore_above": 256,
            "type": "keyword"
          }
        },
        "type": "text"
      },
      "buildAction": {
        "fields": {
          "keyword": {
            "ignore_above": 256,
            "type": "keyword"
          }
        },
        "type": "text"
      }
    }
  }
}

where location is geo_point. In your case, it sounds like the index may have already been created before this API call was made, possibly as a result of indexing a document before the explicit mapping for location was created. In this case, a mapping cannot be updated to be changed from one field datatype to another.

To resolve, you would need to delete the index and create it again, using a create index API call like above.

Than you for your response, I tried your example, but I can't get valid response from searching geodistance, it return "Invalid response" ,cause of problem is geolocation

Can you share the .DebugInformation on the response?

DebugInformation:
"Invalid NEST response built from a unsuccessful low level call on POST: /honadona/_search?typed_keys=true\n# Audit trail of this API call:\n - [1] BadResponse: Node: https://search-honadon-aws-s2kjvr++++++++++++++++++.us-east-2.es.amazonaws.com/ Took: 00:00:02.9632510\n# OriginalException: Elasticsearch.Net.ElasticsearchClientException: Request failed to execute. Call: Status code 404 from: POST /honadona/_search?typed_keys=true. ServerError: Type: index_not_found_exception Reason: "no such index"\n# Request:\r\n<Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>\n# Response:\r\n<Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>\n"

The index honadona does not exist in the cluster. When you create the index, can you post the .DebugInformation returned?

Sorry this is my fault, I have deleted my index and forgot about it , now i reindexed , and search with geodistance and get this error in .DebugInformation:

"Invalid NEST response built from a unsuccessful low level call on POST: /honadona/_search?typed_keys=true\n# Audit trail of this API call:\n - [1] BadResponse: Node: https://search-honadon-aws-s2kjvr4n5nc++++++++.us-east-2.es.amazonaws.com/ Took: 00:00:00.2076510\n# OriginalException: Elasticsearch.Net.ElasticsearchClientException: Request failed to execute. Call: Status code 400 from: POST /honadona/_search?typed_keys=true. ServerError: Type: search_phase_execution_exception Reason: "all shards failed"\n# Request:\r\n{"from":0,"query":{"bool":{"filter":[{"geo_distance":{"distance":"4km","location":{"lat":41.23,"lon":69.32}}}]}},"size":600}\n# Response:\r\n{"error":{"root_cause":[{"type":"query_shard_exception","reason":"failed to find geo_point field [location]","index_uuid":"sHpZSzf1Qq-vsA15MdSGcg","index":"honadona"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"honadona","node":"C92VzD0wSwKIeO5m0bm1cw","reason":{"type":"query_shard_exception","reason":"failed to find geo_point field [location]","index_uuid":"sHpZSzf1Qq-vsA15MdSGcg","index":"honadona"}}]},"status":400}\n"

Can you share the mapping in the honadona index?

Did you mean this:

 public class ElasticSearchModel : Model<Building>
    {     
         public ElasticSearchModel()
        {
            BuildingRooms = new List<BuildingRoomViewModel>();
            BuildingPhotos = new List<BuildingPhotoViewModel>();
         
        }
        public GeoLocation Location { get; set;}
        public int Id { get; set; }
        public string BuildTitle { get; set; }
        public DateTime CreatedDate { get; set; }
        public int TotalRooms { get; set; }
        public string Description { get; set; }
        public decimal? OwnerPrice { get; set; }
        public double? Size { get; set; }
        public string SizeName { get; set; }
        public string MoneyType { get; set; }
        public string BuildAction { get; set; }
        public string BuildType { get; set; }
        public string Address { get; set; }
        public IList<BuildingRoomViewModel> BuildingRooms { get; set; }
        public IList<BuildingPhotoViewModel> BuildingPhotos { get; set; }

         public override void LoadEntity(Building t)
        {
            base.LoadEntity(t);

            SizeName = t.SizeName?.Localizations?.GetSingleOrDefaultByCurrentCulture().Name;
            MoneyType = t.MoneyType?.Localizations?.GetSingleOrDefaultByCurrentCulture().Name;
            BuildAction = t.BuildAction?.Localizations?.GetSingleOrDefaultByCurrentCulture().Name;
            BuildType = t.BuildType?.Localizations?.GetSingleOrDefaultByCurrentCulture().Name;

            // foreach (var item in BuildingRooms)
            // {
            //     item.Name = RoomService.Instance.GetOne(item.RoomType).Value;
            //     item.Assigned = true;
            // }
        }
     

    }
}

No. If you do

GET /honadona

to get the mappings in the honadona index in Elasticsearch, what does it return?

{

* "honadona": {
  * "aliases": { },
  * "mappings": {
    * "_doc": {
      * "properties": {
        * "address": {
          * "type": "text",
          * "fields": {
            * "keyword": {
              * "type": "keyword",
              * "ignore_above": 256}}},
        * "buildAction": {
          * "type": "text",
          * "fields": {
            * "keyword": {
              * "type": "keyword",
              * "ignore_above": 256}}},
        * "buildTitle": {
          * "type": "text",
          * "fields": {
            * "keyword": {
              * "type": "keyword",
              * "ignore_above": 256}}},
        * "buildType": {
          * "type": "text",
          * "fields": {
            * "keyword": {
              * "type": "keyword",
              * "ignore_above": 256}}},
        * "buildingPhotos": {
          * "properties": {
            * "path": {
              * "type": "text",
              * "fields": {
                * "keyword": {
                  * "type": "keyword",
                  * "ignore_above": 256}}},
            * "title": {
              * "type": "text",
              * "fields": {
                * "keyword": {
                  * "type": "keyword",
                  * "ignore_above": 256}}}}},
        * "buildingRooms": {
          * "properties": {
            * "roomCount": {
              * "type": "long"},
            * "roomType": {
              * "type": "long"}}},
        * "createdDate": {
          * "type": "date"},
        * "description": {
          * "type": "text",
          * "fields": {
            * "keyword": {
              * "type": "keyword",
              * "ignore_above": 256}}},
        * "id": {
          * "type": "long"},
        * "location": {
          * "properties": {
            * "lat": {
              * "type": "float"},
            * "lon": {
              * "type": "float"}}},
        * "moneyType": {
          * "type": "text",
          * "fields": {
            * "keyword": {
              * "type": "keyword",
              * "ignore_above": 256}}},
        * "ownerPrice": {
          * "type": "float"},
        * "size": {
          * "type": "float"},
        * "sizeName": {
          * "type": "text",
          * "fields": {
            * "keyword": {
              * "type": "keyword",
              * "ignore_above": 256}}},
        * "totalRooms": {
          * "type": "long"}}}},
  * "settings": {
    * "index": {
      * "creation_date": "1560402955196",
      * "number_of_shards": "5",
      * "number_of_replicas": "1",
      * "uuid": "sHpZSzf1Qq-vsA15MdSGcg",
      * "version": {
        * "created": "6050499"},
      * "provided_name": "honadona"}}}

}

Did you mean this?

Yes. As we can see from the mapping

location is mapped as an object type with lat and lon properties of type float, which is not what we want.

location must be mapped as a geo_point type to perform a geo distance query on the field. You won't be able to change the existing location field mapping type from object to geo_point, so you would either need to

  1. Introduce a new field into the mapping, mapped as a geo_point

or

  1. Create a new index with the location mapped as a geo_point as in Nest elasticsearch 7 Geolocation indexing - #2 by forloop then either,

    a) index documents into this new index

    or

    b) reindex documents from the existing index into this new index.

Please note that you should explicitly create the index and mapping before indexing any documents into it, otherwise Elasticsearch will create the index and infer the mapping based on the first document it sees. Mapping inference will not infer the correct mapping for location, which may possibly be what is happening in your case. You can prevent dynamic field creation by setting dynamic: false on index mappings, and can apply this as a convention to all created indices with index templates.

I tried to fix my problem, but now I have this problem:

"Invalid NEST response built from a unsuccessful low level call on PUT: /honadonz\n# Audit trail of this API call:\n - [1] BadResponse: Node: https://search-honadon-aws-s2kjvr4n5n+++++++++jw2e.us-east-2.es.amazonaws.com/ Took: 00:00:04.7946450\n# OriginalException: Elasticsearch.Net.ElasticsearchClientException: Request failed to execute. Call: Status code 400 from: PUT /honadonz. ServerError: Type: mapper_parsing_exception Reason: "Failed to parse mapping [properties]: Root mapping definition has unsupported parameters: [createdDate : {type=date}] [size : {type=double}] [sizeName : {fields={keyword={ignore_above=256, type=keyword}}, type=text}] [ownerPrice : {type=double}] [buildAction : {fields={keyword={ignore_above=256, type=keyword}}, type=text}] [moneyType : {fields={keyword={ignore_above=256, type=keyword}}, type=text}] [description : {fields={keyword={ignore_above=256, type=keyword}}, type=text}] [location : {type=geo_point}] [id : {type=integer}] [buildTitle : {fields={keyword={ignore_above=256, type=keyword}}, type=text}] [totalRooms : {type=integer}]" CausedBy: "Type: mapper_parsing_exception Reason: "Root mapping definition has unsupported parameters: [createdDate : {type=date}] [size : {type=double}] [sizeName : {fields={keyword={ignore_above=256, type=keyword}}, type=text}] [ownerPrice : {type=double}] [buildAction : {fields={keyword={ignore_above=256, type=keyword}}, type=text}] [moneyType : {fields={keyword={ignore_above=256, type=keyword}}, type=text}] [description : {fields={keyword={ignore_above=256, type=keyword}}, type=text}] [location : {type=geo_point}] [id : {type=integer}] [buildTitle : {fields={keyword={ignore_above=256, type=keyword}}, type=text}] [totalRooms : {type=integer}]""\n# Request:\r\n<Request stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>\n# Response:\r\n<Response stream not captured or already read to completion by serializer. Set DisableDirectStreaming() on ConnectionSettings to force it to be set on the response.>\n"

What version of Elasticsearch are you running against? Note that major versions of NEST are only compatible with major versions of Elasticsearch

  • NEST 5.x -> Elasticsearch 5.x
  • NEST 6.x -> Elasticsearch 6.x
  • NEST 7.x -> Elasticsearch 7.x

So be sure that you're using the correct major version of NEST for the version of Elasticsearch you're targeting. It appears you may be running against Amazon Elasticsearch, which I don't believe supports 7.x, so this may be why the call is failing.

To be sure, can you set .DisableDirectStreaming() on ConnectionSettings

var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));

var settings = new ConnectionSettings(pool)
    .DisableDirectStreaming()
    .PrettyJson();

var client = new ElasticClient(settings);

or on the specific request

var createIndexResponse = client.CreateIndex("honadona", c => c
	.Map<ElasticSearchModel>(mm => mm
		.AutoMap()
	)
    .RequestConfiguration(r => r
        .DisableDirectStreaming()
    )
);

so that you can capture the request and the response, and post the .DebugInformation here?

Thank you very much, now it's working. My elasticsearch version was 7 and Amazon Elasticsearch version was 6.5, and I have changed my elasticsearch to 6.5. and this solved my problem.
Thank you!!!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.