Geo_shape filter in post_filter

HeonKyu_Park · February 12, 2014, 7:11am

I have 2 types of document as below.

poi: 4 millions, it has a point(lon, lat) field
region: it has a polygon field

I'd like to search POI documents in a certain region by using geo_shape
filter.

So I made query as below.

{
"query": {
"filtered": {
"query": {
"match_all": {}
}
, "filter": {
"and": [
{"term": {"properties.category1":"2"}}
, {"term": {"properties.category2":"1"}}
, {"term": {"properties.category3":"1"}}
, {"prefix": {"title":"xxx"}}
]
}
}
}
, "post_filter": {
"geo_shape": {
"geometry": {
"indexed_shape": {
"id": "21000",
"type": "region",
"index": "test",
"shape_field_name": "geometry"
}
}
}
}
}

The count of POI that's title contains 'xxx' is* only 4.*

But the response time of this query was 20 seconds.

so I tested this query without the geo_shape and post_filter.

{
"query": {

  "filtered": {
    "query": {
      "match_all": {}
    }
    , "filter": {
      "and": [ 
        {"term": {"properties.category1":"2"}}
        , {"term": {"properties.category2":"1"}}
        , {"term": {"properties.category3":"1"}}
        , {"prefix": {"title":"xxx"}} 
      ]
    }
  }
}

}

Result count of this query was only 4 and It took only 10 ms.

I though that the number of POI documents that are applied to geo_shape
filter is only 4 by using post_filter.

Why post_filter + geo_shape filters are so slow?

Did I use the post_filter + geo_shape query correctlly?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6f578e25-beef-4c11-a913-605834fe4dae%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Binh_Ly · February 12, 2014, 4:38pm

I'm very curious, does this query make any difference in terms of speed
(might need to run this more than once)?

{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"and": [
{
"bool": {
"must": [
{
"term": {
"properties.category1": "2"
}
},
{
"term": {
"properties.category2": "1"
}
},
{
"term": {
"properties.category3": "1"
}
},
{
"prefix": {
"title": "xxx"
}
}
]
}
},
{
"geo_shape": {
"geometry": {
"indexed_shape": {
"id": "21000",
"type": "region",
"index": "test",
"shape_field_name": "geometry"
}
}
}
}
]
}
}
}
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/01bec9b4-bd20-422e-a5ec-251ccd442d7c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

HeonKyu_Park · February 13, 2014, 3:39am

Thanks for your response. Binh.

No difference. It still took 20 seconds.

There was no improvement on speed.

2014년 2월 13일 목요일 오전 1시 38분 51초 UTC+9, Binh Ly 님의 말:

I'm very curious, does this query make any difference in terms of speed
(might need to run this more than once)?

{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"and": [
{
"bool": {
"must": [
{
"term": {
"properties.category1": "2"
}
},
{
"term": {
"properties.category2": "1"
}
},
{
"term": {
"properties.category3": "1"
}
},
{
"prefix": {
"title": "xxx"
}
}
]
}
},
{
"geo_shape": {
"geometry": {
"indexed_shape": {
"id": "21000",
"type": "region",
"index": "test",
"shape_field_name": "geometry"
}
}
}
}
]
}
}
}
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e3ae3dad-1c62-443e-800d-c45021e0f6ac%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

chilling · February 14, 2014, 2:47am

hi HeonKyu,

may I ask, what your polygons look like. Do they contain wholes? How many
points describing your polygons? Maybe the geo_polygon_filter is an
alternative for you.

Thanks,
Florian

On Thursday, February 13, 2014 12:39:45 PM UTC+9, HeonKyu Park wrote:

Thanks for your response. Binh.

No difference. It still took 20 seconds.

There was no improvement on speed.

2014년 2월 13일 목요일 오전 1시 38분 51초 UTC+9, Binh Ly 님의 말:

I'm very curious, does this query make any difference in terms of speed
(might need to run this more than once)?

{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"and": [
{
"bool": {
"must": [
{
"term": {
"properties.category1": "2"
}
},
{
"term": {
"properties.category2": "1"
}
},
{
"term": {
"properties.category3": "1"
}
},
{
"prefix": {
"title": "xxx"
}
}
]
}
},
{
"geo_shape": {
"geometry": {
"indexed_shape": {
"id": "21000",
"type": "region",
"index": "test",
"shape_field_name": "geometry"
}
}
}
}
]
}
}
}
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f088909f-feff-413d-bfee-3175ae7722ad%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

HeonKyu_Park · February 14, 2014, 5:12am

Hi Forian,

The polygon consists of 2,000 points and it's not multi-polygon. (no
interior-ring, no exterior-ring).

and I will test geo polygon filter and let you know soon.

but I don't want to read the polygon data to pass to geo polygon filter
because it's too big.That's why I use geoshape filter.

Thanks

2014년 2월 14일 금요일 오전 11시 47분 53초 UTC+9, Florian Schilling 님의 말:

hi HeonKyu,

may I ask, what your polygons look like. Do they contain wholes? How many
points describing your polygons? Maybe the geo_polygon_filter is an
alternative for you.

Thanks,
Florian

On Thursday, February 13, 2014 12:39:45 PM UTC+9, HeonKyu Park wrote:

Thanks for your response. Binh.

No difference. It still took 20 seconds.

There was no improvement on speed.

2014년 2월 13일 목요일 오전 1시 38분 51초 UTC+9, Binh Ly 님의 말:

I'm very curious, does this query make any difference in terms of speed
(might need to run this more than once)?

{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"and": [
{
"bool": {
"must": [
{
"term": {
"properties.category1": "2"
}
},
{
"term": {
"properties.category2": "1"
}
},
{
"term": {
"properties.category3": "1"
}
},
{
"prefix": {
"title": "xxx"
}
}
]
}
},
{
"geo_shape": {
"geometry": {
"indexed_shape": {
"id": "21000",
"type": "region",
"index": "test",
"shape_field_name": "geometry"
}
}
}
}
]
}
}
}
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/980999f6-b06a-4d96-8b17-5c25d585ffc2%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

chilling · February 14, 2014, 7:46am

Hi HeonKyu,

I also prefer the geo_shape filter for huge polygons. I'm going to improve
the code there.

Thanks,
Florian

On Friday, February 14, 2014 2:12:29 PM UTC+9, HeonKyu Park wrote:

Hi Forian,

The polygon consists of 2,000 points and it's not multi-polygon. (no
interior-ring, no exterior-ring).

and I will test geo polygon filter and let you know soon.

but I don't want to read the polygon data to pass to geo polygon filter
because it's too big.That's why I use geoshape filter.

Thanks

2014년 2월 14일 금요일 오전 11시 47분 53초 UTC+9, Florian Schilling 님의 말:

hi HeonKyu,

may I ask, what your polygons look like. Do they contain wholes? How many
points describing your polygons? Maybe the geo_polygon_filter is an
alternative for you.

Thanks,
Florian

On Thursday, February 13, 2014 12:39:45 PM UTC+9, HeonKyu Park wrote:

Thanks for your response. Binh.

No difference. It still took 20 seconds.

There was no improvement on speed.

2014년 2월 13일 목요일 오전 1시 38분 51초 UTC+9, Binh Ly 님의 말:

I'm very curious, does this query make any difference in terms of speed
(might need to run this more than once)?

{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"and": [
{
"bool": {
"must": [
{
"term": {
"properties.category1": "2"
}
},
{
"term": {
"properties.category2": "1"
}
},
{
"term": {
"properties.category3": "1"
}
},
{
"prefix": {
"title": "xxx"
}
}
]
}
},
{
"geo_shape": {
"geometry": {
"indexed_shape": {
"id": "21000",
"type": "region",
"index": "test",
"shape_field_name": "geometry"
}
}
}
}
]
}
}
}
}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d4139d74-3f1a-4f39-b332-3644aff58dad%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.