How to improve my GeoPoint ElasticSearch Index for better response time?

Elastic Stack Elasticsearch
#1

Below is my Index (tz_geo) and mapping details:

Index setting:

{
  "index.blocks.read_only_allow_delete": "false",
  "index.priority": "1",
  "index.query.default_field": [
    "*"
  ],
  "index.write.wait_for_active_shards": "1",
  "index.refresh_interval": "30s",
  "index.number_of_replicas": "7"
}

Mapping:

{
  "mapping": {
    "properties": {     
      "geo": {
        "type": "geo_point"
      },
      "gnameid": {
        "type": "long",
        "index": false
      },
      "iso": {
        "type": "keyword",
        "index": false
      },
      "locid": {
        "type": "long",
        "index": false
      },      
      "tz": {
        "type": "keyword",
        "index": false
      }
    }
  }
}

ElasticSearch details:

{
  "name" : "node-1",
  "cluster_name" : "test-cluster",
  "cluster_uuid" : "ET-Gwb-iR4mCmzZUppuOKw",
  "version" : {
    "number" : "7.2.0",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "508c38a",
    "build_date" : "2019-06-20T15:54:18.811730Z",
    "build_snapshot" : false,
    "lucene_version" : "8.0.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Number of nodes in cluster: 11

Index data size: 9.6 GB

Total documents in Index: 12025944

My Geo Search using C# ElasticSearch NEST

var result = await ES_Client.SearchAsync<ES_TZGeo>(s => s
                   .Index("tz_geo")                   
                   .Query(query => query.Bool(b => b.Filter(filter => filter
                   .GeoDistance(geo => geo
                       .Field(f => f.geo)
                       .Distance("500km").Location(lat, lon)
                       .DistanceType(GeoDistanceType.Arc)

                   )

                   )

                   )

                )
                   .Sort(sort => sort
                       .GeoDistance(g => g
                           .Field(f => f.geo)
                           .Order(SortOrder.Ascending)
                           .Unit(DistanceUnit.Kilometers)
                           .Points(new GeoLocation(lat, lon))
                           .DistanceType(GeoDistanceType.Arc)))
                           .Size(1)
                   );

The above search will find the nearest document based on lat,lon provided.

We are seeing that sometimes the response is more than 1 sec for the above query. Is there any recommendation on how to improve on this search either in terms of NEST or maybe change the replicas, etc?

Thank you for your time.

#2

The performance of this query should improve if you change the units for sorting to be meters instead of kilometers.

1 Like
#3

Will I also need to change the Query part and send the distance as meter instead of km or is it just for sorting?

Thanks for your reply.

#4

just sorting

#5

Thank you for your answer. Really appreciate it.

May I know what is the reasoning behind this? Is there anywhere I could read more about this?

Also, if you have time, will changing the replica shard to higher values help improve the search?

Thank you.

#6

The reason is a limitation in an efficient lucene algorithm to perform sorting as it only works when distance units are in meters:

If units are different than meters, we use a naive approach where we calculate the distance of all points in the index which is less efficient.

Increase the number of replicas should not improve the performance of the query but it will help if you have many concurrent requests. How many shards do you have in the index?

1 Like
#7

For this index it is set with 1 primary and 7 replica for a 11 node cluster.

Any suggestions to improve this?

#8

I expect this topology to work on a high concurrency environment. For the size of the index I think having one primary is fine.

You might win a bit of performance having one more primary but I am hesitant to recommend that. I think the only way to know for sure if it helps your case is by testing with a realistic workload.

by the way, did it help the advise above? Could you share the numbers?

#9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.