Elasticsearch geo search strange behavior

Elastic Stack Elasticsearch
1

A few days ago I faced with the strange behavior of geo search in Elasticsearch.

I use AWS managed ES 5.5, obviously over REST interface.

Assume we have 200k objects with location info represented as the point only. I use geo search to find the points within multiple polygons. They are shown on the image below. Coordinates were extracted from final request to the ES.

The request is built using official Java High-level REST client. The request query will be attached below.

I want to search for all objects within at least one polygon.
Here is the query (real fields names and values were replaced by stub, Except location, and locationPoint.coordinates)

Query is here --> https://goo.gl/3W62kA

Note, that field location is mapped as geo_shape and field location.coordinates is mapped as geo_point.

So the problem is next. Below the results (hits count) of requests are presented. Only polygons are changing.

#  Polygons                              Hits count

1) 1,2,3,4								 5565

2) 1									 4897

3) 3,4	     							 75

4) 2        							 9

5) 1,3,4        						 5543

6) 1,2              					 5466

7) 2,3,4            					 84

So, if I add results of polygon 1st with 2,3,4 polygons I will not obtain the number as it was in full request.

For example, #1 != #2 + #7, also #1 != #5 + #4, but #7 == #4 + #3

I cannot understand whether it is the issue in this request or expected behavior or even bug in ES.

Can anyone help me to understand the logic of such ES behavior or point to the solution?

Thanks!

2

can you replicate this issue with vanilla elasticsearch or with an instance running on elastic cloud [1]?. The AWS is not managed or run by us and it might run non-vanilla ES instances.

[1] https://www.elastic.co/cloud

3

Hi Simon,

Unfortunately, I cannot do it due to the large amount of specific data that should not be copied anywhere except AWS.
Are you talking about some fork of vanilla ES?
AFAIK, Amazon provides vanilla ES just with some restrictions on settings and access to raw files.

4

can you paste me the output of http://yourelasticsearchip:9200

5

yeah so Elastic Cloud: Hosted Elasticsearch, Hosted Search | Elastic runs on AWS as well.

6

Here is the response:
{
"name": "zFHTplx",
"cluster_name": "889835723148:disco-es",
"cluster_uuid": "WtJtJkRdRpyzndcsF7-G4w",
"version": {
"number": "5.5.2",
"build_hash": "Unknown",
"build_date": "2017-10-18T04:35:01.381Z",
"build_snapshot": false,
"lucene_version": "6.6.0"
},
"tagline": "You Know, for Search"
}

yeah so Elastic Cloud: Hosted Elasticsearch, Hosted Search | Elastic runs on AWS as well.

I meant that I cannot do it due to security policies inside the company.

7 
{
  "name" : "3E1A4vn",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "ZWnDTtaoQxiur9tlYaFslA",
  "version" : {
    "number" : "5.5.2",
    "build_hash" : "b2f0c09",
    "build_date" : "2017-08-14T12:33:14.154Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.0"
  },
  "tagline" : "You Know, for Search"
}

This is the repsonse from a vanilla 5.5.2 (see build hash and date). You are running a modified version of elasticsearch, I can't help you with your questions from AWS elasticsearch you need to get help there or replicate it on a vanilla version. I am sorry about that.

I meant that I cannot do it due to security policies inside the company.

What policies are that I mean elastic cloud runs on the same infrastructure as the service that you are using.

8

Thank you for help. Will try to find solution on AWS discuss.

Have a nice day!

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.