Spatial query against indexed bounding boxes

Elastic Stack Elasticsearch
1

Hi,

I noticed only a few days ago that the only "spatial" type available
is a geo_point (which is already good!).
Yet, that makes spatial query quite limited.

Are there some plan about making a bouding box or a polygon as new
types with associated queries (contains/intersects) ?
It would be awesome.

Thanks a lot,

Simon

2

The geo_point type is just there to mark the field as geo. You already
have distance, bbox, and poly geo filters, sort by geo distance, and geo
distance faceting.

On Tue, Jan 24, 2012 at 11:06 AM, djcoin [Simon Thépot] <
simon.thepot@gmail.com> wrote:

Hi,

I noticed only a few days ago that the only "spatial" type available
is a geo_point (which is already good!).
Yet, that makes spatial query quite limited.

Are there some plan about making a bouding box or a polygon as new
types with associated queries (contains/intersects) ?
It would be awesome.

Thanks a lot,

Simon

3

Thanks for your quick reply,

I saw I can use all those filters.
But I did not see any examples where a spatial query get applied to a
geometry that is not a point on the server side.

Eg: Is there a way to index Polygone/Line in ES and make a bbox
request on it, looking for an intersection between the bounding box
used in the filter and the 'non-point' geometry stored on the server
side ?

Thanks again,
Simon

On 24 jan, 15:58, Shay Banon kim...@gmail.com wrote:

The geo_point type is just there to mark the field as geo. You already
have distance, bbox, and poly geo filters, sort by geo distance, and geo
distance faceting.

On Tue, Jan 24, 2012 at 11:06 AM, djcoin [Simon Thépot] <

simon.the...@gmail.com> wrote:

Hi,

I noticed only a few days ago that the only "spatial" type available
is a geo_point (which is already good!).
Yet, that makes spatial query quite limited.

Are there some plan about making a bouding box or a polygon as new
types with associated queries (contains/intersects) ?
It would be awesome.

Thanks a lot,

Simon

4

No, there isn't support for indexing more complex data structures aside from geo points, though, if you can structure it in a way that is represented by several geo points, and then use geo point level search (combined with and/or) it might be good enough.

On Tuesday, January 24, 2012 at 10:07 PM, djcoin [Simon Thépot] wrote:

Thanks for your quick reply,

I saw I can use all those filters.
But I did not see any examples where a spatial query get applied to a
geometry that is not a point on the server side.

Eg: Is there a way to index Polygone/Line in ES and make a bbox
request on it, looking for an intersection between the bounding box
used in the filter and the 'non-point' geometry stored on the server
side ?

Thanks again,
Simon

On 24 jan, 15:58, Shay Banon <kim...@gmail.com (http://gmail.com)> wrote:

The geo_point type is just there to mark the field as geo. You already
have distance, bbox, and poly geo filters, sort by geo distance, and geo
distance faceting.

On Tue, Jan 24, 2012 at 11:06 AM, djcoin [Simon Thépot] <

simon.the...@gmail.com (http://gmail.com)> wrote:

Hi,

I noticed only a few days ago that the only "spatial" type available
is a geo_point (which is already good!).
Yet, that makes spatial query quite limited.

Are there some plan about making a bouding box or a polygon as new
types with associated queries (contains/intersects) ?
It would be awesome.

Thanks a lot,

Simon

5

I work on a NASA project that's using Elasticsearch and just went
operational earlier this month. The earth science data we index is
usually polygons and bounding boxes. We combined Elasticsearch with
Oracle Spatial as a full solution. When we index an object with a
spatial area we find the minimum bounding rectangle (MBR) of the
spatial area. The MBR consists of north, south, east and west
coordinates. The MBR gets put in Elasticsearch and we put the real
spatial area in Oracle Spatial. During a search from our
application we first convert the user's search area to an MBR. We
send the MBR with other search terms to elastic. Elastic search
returns very quickly and all of the results are very likely spatial
matches because the minimum bounding rectangles intersect. We then do
real spatial area intersections in Oracle to get exact matches. We
usually end up with about 80-100% of the results from elastic actually
intersecting the user's search area. We still need Oracle Spatial to
do the real spatial intersections but our searches got about twice as
fast overall when we switched from Oracle to Elasticsearch.

On Jan 25, 11:26 am, Shay Banon kim...@gmail.com wrote:

No, there isn't support for indexing more complex data structures aside from geo points, though, if you can structure it in a way that is represented by several geo points, and then use geo point level search (combined with and/or) it might be good enough.

On Tuesday, January 24, 2012 at 10:07 PM, djcoin [Simon Thépot] wrote:

Thanks for your quick reply,

I saw I can use all those filters.
But I did not see any examples where a spatial query get applied to a
geometry that is not a point on the server side.

Eg: Is there a way to index Polygone/Line in ES and make a bbox
request on it, looking for an intersection between the bounding box
used in the filter and the 'non-point' geometry stored on the server
side ?

Thanks again,
Simon

On 24 jan, 15:58, Shay Banon <kim...@gmail.com (http://gmail.com)> wrote:

The geo_point type is just there to mark the field as geo. You already
have distance, bbox, and poly geo filters, sort by geo distance, and geo
distance faceting.

On Tue, Jan 24, 2012 at 11:06 AM, djcoin [Simon Thépot] <

simon.the...@gmail.com (http://gmail.com)> wrote:

Hi,

I noticed only a few days ago that the only "spatial" type available
is a geo_point (which is already good!).
Yet, that makes spatial query quite limited.

Are there some plan about making a bouding box or a polygon as new
types with associated queries (contains/intersects) ?
It would be awesome.

Thanks a lot,

Simon

6

Hey, thanks for the answers/highlights!

At the moment, my only need is filtering on intersection between a
bbox (input of the user) and edges.
As I export the data from PosgreSQL/PostGIS to ES, I planned me too to
have an hybrid solution, with spatial specific query being fed to the
spatial database.
Another solution I was thinking about for my case was to cut the edges
in smaller edges (given some criteria, eg: every X kms), thus yielding
'geo_point' that I would be able to query against with a sastisfying
precision in ES.

Jason, I'm curious, how do you fed to Oracle the result of your first
filter ? You may obtain thousand of result matching the ES query, do
you actually make some 'SELECT .. FROM .. Where id in [<.. your
thousand/millions results>]'.
I did not try it myself, but I guess that may exceed the maximum
length of and SQL query.

And second question ! :slight_smile: How do calculate intersection between two MBR
in ES (user input and stored data) ?

Thanks a lot !
Simon

On 26 jan, 13:19, Jason ja...@element84.com wrote:

I work on a NASA project that's using Elasticsearch and just went
operational earlier this month. The earth science data we index is
usually polygons and bounding boxes. We combined Elasticsearch with
OracleSpatialas a full solution. When we index an object with aspatialarea we find the minimum bounding rectangle (MBR) of thespatialarea. The MBR consists of north, south, east and west
coordinates. The MBR gets put in Elasticsearch and we put the realspatialarea in OracleSpatial. During a search from our
application we first convert the user's search area to an MBR. We
send the MBR with other search terms to elastic. Elastic search
returns very quickly and all of the results are very likelyspatial
matches because the minimum bounding rectangles intersect. We then do
realspatialarea intersections in Oracle to get exact matches. We
usually end up with about 80-100% of the results from elastic actually
intersecting the user's search area. We still need OracleSpatialto
do the realspatialintersections but our searches got about twice as
fast overall when we switched from Oracle to Elasticsearch.

On Jan 25, 11:26 am, Shay Banon kim...@gmail.com wrote:

No, there isn't support for indexing more complex data structures aside from geo points, though, if you can structure it in a way that is represented by several geo points, and then use geo point level search (combined with and/or) it might be good enough.

On Tuesday, January 24, 2012 at 10:07 PM, djcoin [Simon Thépot] wrote:

Thanks for your quick reply,

I saw I can use all those filters.
But I did not see any examples where aspatialquery get applied to a
geometry that is not a point on the server side.

Eg: Is there a way to index Polygone/Line in ES and make a bbox
request on it, looking for an intersection between the bounding box
used in the filter and the 'non-point' geometry stored on the server
side ?

Thanks again,
Simon

On 24 jan, 15:58, Shay Banon <kim...@gmail.com (http://gmail.com)> wrote:

The geo_point type is just there to mark the field as geo. You already
have distance, bbox, and poly geo filters, sort by geo distance, and geo
distance faceting.

On Tue, Jan 24, 2012 at 11:06 AM, djcoin [Simon Thépot] <

simon.the...@gmail.com (http://gmail.com)> wrote:

Hi,

I noticed only a few days ago that the only "spatial" type available
is a geo_point (which is already good!).
Yet, that makesspatialquery quite limited.

Are there some plan about making a bouding box or a polygon as new
types with associated queries (contains/intersects) ?
It would be awesome.

Thanks a lot,

Simon

7

We can only return a maximum of 2000 results to the user on our API.
We retrieve something like ~2,500 matching results from elastic and
then query oracle for exact spatial matches using sql like "select *
from spatial_table where and (id in (<1000 ids>)
or id in (<1000 ids>) or id in ())" We'll usually only
need to execute this single statement given that we've already done
MBR filtering in elastic. As long as we've found 2000 ids or more
from Oracle we'll have found enough to return to the user. We get an
extra 500 or so from elastic to help ensure we'll only need to query
oracle one time.

I don't think I can put the exact algorithm we use to intersect
bounding rectangles in elastic but bounding rectangle intersections is
pretty simple compared to regular spatial intersections. You should
think of the a bounding rectangle as two ranges. One is south to
north, the latitude range, and the other is west to east, the
longitude range. The two minimum bounding rectangles intersect if
both latitude ranges intersect and both longitude ranges intersect. A
range intersection is a little trickier with elastic given that
elastic doesn't have < and > operators like a traditional database.
It has a range query (Elasticsearch Platform — Find real-time answers at scale | Elastic
query-dsl/range-query.html) which made it harder for me to get. It
helps to draw examples when you come up with the exact algorithm.

On Jan 27, 3:56 am, djcoin [Simon Thépot] simon.the...@gmail.com
wrote:

Hey, thanks for the answers/highlights!

At the moment, my only need is filtering on intersection between a
bbox (input of the user) and edges.
As I export the data from PosgreSQL/PostGIS to ES, I planned me too to
have an hybrid solution, with spatial specific query being fed to the
spatial database.
Another solution I was thinking about for my case was to cut the edges
in smaller edges (given some criteria, eg: every X kms), thus yielding
'geo_point' that I would be able to query against with a sastisfying
precision in ES.

Jason, I'm curious, how do you fed to Oracle the result of your first
filter ? You may obtain thousand of result matching the ES query, do
you actually make some 'SELECT .. FROM .. Where id in [<.. your
thousand/millions results>]'.
I did not try it myself, but I guess that may exceed the maximum
length of and SQL query.

And second question ! :slight_smile: How do calculate intersection between two MBR
in ES (user input and stored data) ?

Thanks a lot !
Simon

On 26 jan, 13:19, Jason ja...@element84.com wrote:

I work on a NASA project that's using Elasticsearch and just went
operational earlier this month. The earth science data we index is
usually polygons and bounding boxes. We combined Elasticsearch with
OracleSpatialas a full solution. When we index an object with aspatialarea we find the minimum bounding rectangle (MBR) of thespatialarea. The MBR consists of north, south, east and west
coordinates. The MBR gets put in Elasticsearch and we put the realspatialarea in OracleSpatial. During a search from our
application we first convert the user's search area to an MBR. We
send the MBR with other search terms to elastic. Elastic search
returns very quickly and all of the results are very likelyspatial
matches because the minimum bounding rectangles intersect. We then do
realspatialarea intersections in Oracle to get exact matches. We
usually end up with about 80-100% of the results from elastic actually
intersecting the user's search area. We still need OracleSpatialto
do the realspatialintersections but our searches got about twice as
fast overall when we switched from Oracle to Elasticsearch.

On Jan 25, 11:26 am, Shay Banon kim...@gmail.com wrote:

No, there isn't support for indexing more complex data structures aside from geo points, though, if you can structure it in a way that is represented by several geo points, and then use geo point level search (combined with and/or) it might be good enough.

On Tuesday, January 24, 2012 at 10:07 PM, djcoin [Simon Thépot] wrote:

Thanks for your quick reply,

I saw I can use all those filters.
But I did not see any examples where aspatialquery get applied to a
geometry that is not a point on the server side.

Eg: Is there a way to index Polygone/Line in ES and make a bbox
request on it, looking for an intersection between the bounding box
used in the filter and the 'non-point' geometry stored on the server
side ?

Thanks again,
Simon

On 24 jan, 15:58, Shay Banon <kim...@gmail.com (http://gmail.com)> wrote:

The geo_point type is just there to mark the field as geo. You already
have distance, bbox, and poly geo filters, sort by geo distance, and geo
distance faceting.

On Tue, Jan 24, 2012 at 11:06 AM, djcoin [Simon Thépot] <

simon.the...@gmail.com (http://gmail.com)> wrote:

Hi,

I noticed only a few days ago that the only "spatial" type available
is a geo_point (which is already good!).
Yet, that makesspatialquery quite limited.

Are there some plan about making a bouding box or a polygon as new
types with associated queries (contains/intersects) ?
It would be awesome.

Thanks a lot,

Simon