BoundingBox Search takes time


(prashant5375) #1

Hi there ,
Actually i have just stated using Elastic Search , and i have a
situation where in i have to search in the Boundary of lat/long .
When i have created the boundary and performed the search using java
code given below.

client.prepareSearch() .setQuery(filteredQuery(textQuery("mappedfsn",
mapped), geoBoundingBoxFilter("location").topLeft(latt+
spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt-
spreadOnLatitude, longii-
spreadOnLongitude))).setSize(NumRes).execute().actionGet();

I found the search taking too much of time (200 mili sec on the Big
Server with 24gb of ram on it and i kept index on the Ram Disk too)
Now if i remove the boundingbox condition i get the result in no time
say 20 mili sec.

Can any body help me on this.

Thanks in advance.
Regards
Prashant


(Shay Banon) #2

You can try and use different execution mode (documented under type at the bottom here http://www.elasticsearch.org/guide/reference/query-dsl/geo-bounding-box-filter.html) to have it indexed instead of done in memory. As explained there, you will need to have the geo_point type mapping index the lat/lon as well in this case.

On Tuesday, January 31, 2012 at 6:39 PM, BeyondLimit wrote:

Hi there ,
Actually i have just stated using Elastic Search , and i have a
situation where in i have to search in the Boundary of lat/long .
When i have created the boundary and performed the search using java
code given below.

client.prepareSearch() .setQuery(filteredQuery(textQuery("mappedfsn",
mapped), geoBoundingBoxFilter("location").topLeft(latt+
spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt-
spreadOnLatitude, longii-
spreadOnLongitude))).setSize(NumRes).execute().actionGet();

I found the search taking too much of time (200 mili sec on the Big
Server with 24gb of ram on it and i kept index on the Ram Disk too)
Now if i remove the boundingbox condition i get the result in no time
say 20 mili sec.

Can any body help me on this.

Thanks in advance.
Regards
Prashant


(Shay Banon) #3

Also note, that having the index in ram drive will not matter that much, since by default the bbox checks are done in memory with values loaded. For indexed type execution, it will help.

On Tuesday, January 31, 2012 at 7:26 PM, Shay Banon wrote:

You can try and use different execution mode (documented under type at the bottom here http://www.elasticsearch.org/guide/reference/query-dsl/geo-bounding-box-filter.html) to have it indexed instead of done in memory. As explained there, you will need to have the geo_point type mapping index the lat/lon as well in this case.

On Tuesday, January 31, 2012 at 6:39 PM, BeyondLimit wrote:

Hi there ,
Actually i have just stated using Elastic Search , and i have a
situation where in i have to search in the Boundary of lat/long .
When i have created the boundary and performed the search using java
code given below.

client.prepareSearch() .setQuery(filteredQuery(textQuery("mappedfsn",
mapped), geoBoundingBoxFilter("location").topLeft(latt+
spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt-
spreadOnLatitude, longii-
spreadOnLongitude))).setSize(NumRes).execute().actionGet();

I found the search taking too much of time (200 mili sec on the Big
Server with 24gb of ram on it and i kept index on the Ram Disk too)
Now if i remove the boundingbox condition i get the result in no time
say 20 mili sec.

Can any body help me on this.

Thanks in advance.
Regards
Prashant


(prashant5375) #4

Hi Thanks for the reply.
But can you share something using java , coz i am new to CURL .
It will be great if u can explain i java language.
Any java example which explains what you are saying.
Regards
Prashant
On Jan 31, 10:27 pm, Shay Banon kim...@gmail.com wrote:

Also note, that having the index in ram drive will not matter that much, since by default the bbox checks are done in memory with values loaded. For indexed type execution, it will help.

On Tuesday, January 31, 2012 at 7:26 PM, Shay Banon wrote:

You can try and use different execution mode (documented under type at the bottom herehttp://www.elasticsearch.org/guide/reference/query-dsl/geo-bounding-b...) to have it indexed instead of done in memory. As explained there, you will need to have the geo_point type mapping index the lat/lon as well in this case.

On Tuesday, January 31, 2012 at 6:39 PM, BeyondLimit wrote:

Hi there ,
Actually i have just stated using Elastic Search , and i have a
situation where in i have to search in the Boundary of lat/long .
When i have created the boundary and performed the search using java
code given below.

client.prepareSearch() .setQuery(filteredQuery(textQuery("mappedfsn",
mapped), geoBoundingBoxFilter("location").topLeft(latt+
spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt-
spreadOnLatitude, longii-
spreadOnLongitude))).setSize(NumRes).execute().actionGet();

I found the search taking too much of time (200 mili sec on the Big
Server with 24gb of ram on it and i kept index on the Ram Disk too)
Now if i remove the boundingbox condition i get the result in no time
say 20 mili sec.

Can any body help me on this.

Thanks in advance.
Regards
Prashant


(prashant5375) #5

hi Shay,
i guess you are talking about like this
String mapping =
XContentFactory.jsonBuilder().startObject().startObject("type")
.startObject("properties").startObject("point").field("type",
"geo_point").field("lat_lon", true).endObject().endObject()
.endObject().endObject().string();
But then also no luck. Can anybody help me out. Explaining better way
of using BoundaryBox search in java.
Regards
Prashant

On Jan 31, 10:33 pm, BeyondLimit prashant.vi...@gmail.com wrote:

Hi Thanks for the reply.
But can you share something using java , coz i am new to CURL .
It will be great if u can explain i java language.
Any java example which explains what you are saying.
Regards
Prashant
On Jan 31, 10:27 pm, Shay Banon kim...@gmail.com wrote:

Also note, that having the index in ram drive will not matter that much, since by default the bbox checks are done in memory with values loaded. For indexed type execution, it will help.

On Tuesday, January 31, 2012 at 7:26 PM, Shay Banon wrote:

You can try and use different execution mode (documented under type at the bottom herehttp://www.elasticsearch.org/guide/reference/query-dsl/geo-bounding-b...) to have it indexed instead of done in memory. As explained there, you will need to have the geo_point type mapping index the lat/lon as well in this case.

On Tuesday, January 31, 2012 at 6:39 PM, BeyondLimit wrote:

Hi there ,
Actually i have just stated using Elastic Search , and i have a
situation where in i have to search in the Boundary of lat/long .
When i have created the boundary and performed the search using java
code given below.

client.prepareSearch() .setQuery(filteredQuery(textQuery("mappedfsn",
mapped), geoBoundingBoxFilter("location").topLeft(latt+
spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt-
spreadOnLatitude, longii-
spreadOnLongitude))).setSize(NumRes).execute().actionGet();

I found the search taking too much of time (200 mili sec on the Big
Server with 24gb of ram on it and i kept index on the Ram Disk too)
Now if i remove the boundingbox condition i get the result in no time
say 20 mili sec.

Can any body help me on this.

Thanks in advance.
Regards
Prashant


(Shay Banon) #6

Whats was not lucky? What did not work?

On Wednesday, February 1, 2012 at 6:31 AM, BeyondLimit wrote:

hi Shay,
i guess you are talking about like this
String mapping =
XContentFactory.jsonBuilder().startObject().startObject("type")
.startObject("properties").startObject("point").field("type",
"geo_point").field("lat_lon", true).endObject().endObject()
.endObject().endObject().string();
But then also no luck. Can anybody help me out. Explaining better way
of using BoundaryBox search in java.
Regards
Prashant

On Jan 31, 10:33 pm, BeyondLimit <prashant.vi...@gmail.com (http://gmail.com)> wrote:

Hi Thanks for the reply.
But can you share something using java , coz i am new to CURL .
It will be great if u can explain i java language.
Any java example which explains what you are saying.
Regards
Prashant
On Jan 31, 10:27 pm, Shay Banon <kim...@gmail.com (http://gmail.com)> wrote:

Also note, that having the index in ram drive will not matter that much, since by default the bbox checks are done in memory with values loaded. For indexed type execution, it will help.

On Tuesday, January 31, 2012 at 7:26 PM, Shay Banon wrote:

You can try and use different execution mode (documented under type at the bottom herehttp://www.elasticsearch.org/guide/reference/query-dsl/geo-bounding-b...) to have it indexed instead of done in memory. As explained there, you will need to have the geo_point type mapping index the lat/lon as well in this case.

On Tuesday, January 31, 2012 at 6:39 PM, BeyondLimit wrote:

Hi there ,
Actually i have just stated using Elastic Search , and i have a
situation where in i have to search in the Boundary of lat/long .
When i have created the boundary and performed the search using java
code given below.

client.prepareSearch() .setQuery(filteredQuery(textQuery("mappedfsn",
mapped), geoBoundingBoxFilter("location").topLeft(latt+
spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt-
spreadOnLatitude, longii-
spreadOnLongitude))).setSize(NumRes).execute().actionGet();

I found the search taking too much of time (200 mili sec on the Big
Server with 24gb of ram on it and i kept index on the Ram Disk too)
Now if i remove the boundingbox condition i get the result in no time
say 20 mili sec.

Can any body help me on this.

Thanks in advance.
Regards
Prashant


(prashant5375) #7

Hi ,
I am writing the problem again :
I have a big server of 25 gb ram on it.
And i have a index of 13 gb . When i am running only textQuery it giving me
results very fast say 10 mili sec.
but when i an using boundary box its taking 200+ mili sec in query
execution.

How i have created the Index given below.
------Indexing------------
/*This is the mapping i have created
*/
String mapping =
XContentFactory.jsonBuilder().startObject().startObject("type")

.startObject("properties").startObject("point").field("type",
"geo_point").field("lat_lon", true).endObject().endObject()
.endObject().endObject().string();

//And this is who i am indexing it

brb.add(client.prepareIndex("test", "type",
j+"").setSource(jsonBuilder().startObject()
.field("recid", tmp[1])
.field("mergefield", tmp[4])
.field("mappedfsn", tmp[0].replaceAll(""", ""))
.startObject("point").field("lat",
Double.parseDouble(lati)).field("lon",
Double.parseDouble(longi)).endObject()
.endObject())
);

------------Search------------
response = client.prepareSearch() // from NY
.setQuery(filteredQuery(termQuery("mappedfsn",mapped),
geoBoundingBoxFilter("point").cache(true).topLeft(latt+ spreadOnLatitude,
longii+ spreadOnLongitude).bottomRight(latt- spreadOnLatitude,
longii-spreadOnLongitude)))
.setSize(NumRes).execute().actionGet();

This is how i am searching. Where i m wrong in this.
Regards
Prashant

On Wed, Feb 1, 2012 at 3:13 PM, Shay Banon kimchy@gmail.com wrote:

Whats was not lucky? What did not work?

On Wednesday, February 1, 2012 at 6:31 AM, BeyondLimit wrote:

hi Shay,
i guess you are talking about like this
String mapping =
XContentFactory.jsonBuilder().startObject().startObject("type")
.startObject("properties").startObject("point").field("type",
"geo_point").field("lat_lon", true).endObject().endObject()
.endObject().endObject().string();
But then also no luck. Can anybody help me out. Explaining better way
of using BoundaryBox search in java.
Regards
Prashant

On Jan 31, 10:33 pm, BeyondLimit prashant.vi...@gmail.com wrote:

Hi Thanks for the reply.
But can you share something using java , coz i am new to CURL .
It will be great if u can explain i java language.
Any java example which explains what you are saying.
Regards
Prashant
On Jan 31, 10:27 pm, Shay Banon kim...@gmail.com wrote:

Also note, that having the index in ram drive will not matter that much,
since by default the bbox checks are done in memory with values loaded. For
indexed type execution, it will help.

On Tuesday, January 31, 2012 at 7:26 PM, Shay Banon wrote:

You can try and use different execution mode (documented under type at the
bottom herehttp://
www.elasticsearch.org/guide/reference/query-dsl/geo-bounding-b...) to
have it indexed instead of done in memory. As explained there, you will
need to have the geo_point type mapping index the lat/lon as well in this
case.

On Tuesday, January 31, 2012 at 6:39 PM, BeyondLimit wrote:

Hi there ,
Actually i have just stated using Elastic Search , and i have a
situation where in i have to search in the Boundary of lat/long .
When i have created the boundary and performed the search using java
code given below.

client.prepareSearch() .setQuery(filteredQuery(textQuery("mappedfsn",
mapped), geoBoundingBoxFilter("location").topLeft(latt+
spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt-
spreadOnLatitude, longii-
spreadOnLongitude))).setSize(NumRes).execute().actionGet();

I found the search taking too much of time (200 mili sec on the Big
Server with 24gb of ram on it and i kept index on the Ram Disk too)
Now if i remove the boundingbox condition i get the result in no time
say 20 mili sec.

Can any body help me on this.

Thanks in advance.
Regards
Prashant


(prashant5375) #8

Hi ,
I am writing the problem again :
I have a big server of 25 gb ram on it.
And i have a index of 13 gb . When i am running only textQuery it giving me
results very fast say 10 mili sec.
but when i an using boundary box its taking 200+ mili sec in query
execution.

How i have created the Index given below.
------Indexing------------
/*This is the mapping i have created
*/
String mapping =
XContentFactory.jsonBuilder().startObject().startObject("type")

.startObject("properties").startObject("point").field("type",
"geo_point").field("lat_lon", true).endObject().endObject()
.endObject().endObject().string();

//And this is who i am indexing it

brb.add(client.prepareIndex("test", "type",
j+"").setSource(jsonBuilder().startObject()
.field("recid", tmp[1])
.field("mergefield", tmp[4])
.field("mappedfsn", tmp[0].replaceAll(""", ""))
.startObject("point").field("lat",
Double.parseDouble(lati)).field("lon",
Double.parseDouble(longi)).endObject()
.endObject())
);

------------Search------------
response = client.prepareSearch() // from NY
.setQuery(filteredQuery(termQuery("mappedfsn",mapped),
geoBoundingBoxFilter("point").cache(true).topLeft(latt+ spreadOnLatitude,
longii+ spreadOnLongitude).bottomRight(latt- spreadOnLatitude,
longii-spreadOnLongitude)))
.setSize(NumRes).execute().actionGet();

This is how i am searching. Where i m wrong in this.
Regards
Prashant

On Wed, Feb 1, 2012 at 3:13 PM, Shay Banon kimchy@gmail.com wrote:

Whats was not lucky? What did not work?

On Wednesday, February 1, 2012 at 6:31 AM, BeyondLimit wrote:

hi Shay,
i guess you are talking about like this
String mapping =
XContentFactory.jsonBuilder().startObject().startObject("type")
.startObject("properties").startObject("point").field("type",
"geo_point").field("lat_lon", true).endObject().endObject()
.endObject().endObject().string();
But then also no luck. Can anybody help me out. Explaining better way
of using BoundaryBox search in java.
Regards
Prashant

On Jan 31, 10:33 pm, BeyondLimit prashant.vi...@gmail.com wrote:

Hi Thanks for the reply.
But can you share something using java , coz i am new to CURL .
It will be great if u can explain i java language.
Any java example which explains what you are saying.
Regards
Prashant
On Jan 31, 10:27 pm, Shay Banon kim...@gmail.com wrote:

Also note, that having the index in ram drive will not matter that much,
since by default the bbox checks are done in memory with values loaded. For
indexed type execution, it will help.

On Tuesday, January 31, 2012 at 7:26 PM, Shay Banon wrote:

You can try and use different execution mode (documented under type at the
bottom herehttp://
www.elasticsearch.org/guide/reference/query-dsl/geo-bounding-b...) to
have it indexed instead of done in memory. As explained there, you will
need to have the geo_point type mapping index the lat/lon as well in this
case.

On Tuesday, January 31, 2012 at 6:39 PM, BeyondLimit wrote:

Hi there ,
Actually i have just stated using Elastic Search , and i have a
situation where in i have to search in the Boundary of lat/long .
When i have created the boundary and performed the search using java
code given below.

client.prepareSearch() .setQuery(filteredQuery(textQuery("mappedfsn",
mapped), geoBoundingBoxFilter("location").topLeft(latt+
spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt-
spreadOnLatitude, longii-
spreadOnLongitude))).setSize(NumRes).execute().actionGet();

I found the search taking too much of time (200 mili sec on the Big
Server with 24gb of ram on it and i kept index on the Ram Disk too)
Now if i remove the boundingbox condition i get the result in no time
say 20 mili sec.

Can any body help me on this.

Thanks in advance.
Regards
Prashant


(Shay Banon) #9

Few problems in your code:

  1. Don't set cache to true, as its meaningless when using something like bounding box since it changes all the time. If you set cache to true, it will need to go over all the docs in order to create a matching doc set for them.

  2. In order to check if the indexed option is faster than the memory one, you need to call type("indexed") on the geo bounding box (or type("memory") which is the default).

On Wednesday, February 1, 2012 at 12:09 PM, PS wrote:

Hi ,
I am writing the problem again :
I have a big server of 25 gb ram on it.
And i have a index of 13 gb . When i am running only textQuery it giving me results very fast say 10 mili sec.
but when i an using boundary box its taking 200+ mili sec in query execution.

How i have created the Index given below.
------Indexing------------
/*This is the mapping i have created
*/
String mapping = XContentFactory.jsonBuilder().startObject().startObject("type")
.startObject("properties").startObject("point").field("type", "geo_point").field("lat_lon", true).endObject().endObject()
.endObject().endObject().string();

//And this is who i am indexing it

brb.add(client.prepareIndex("test", "type", j+"").setSource(jsonBuilder().startObject()
.field("recid", tmp[1])
.field("mergefield", tmp[4])
.field("mappedfsn", tmp[0].replaceAll(""", ""))
.startObject("point").field("lat", Double.parseDouble(lati)).field("lon", Double.parseDouble(longi)).endObject()
.endObject())
);

------------Search------------
response = client.prepareSearch() // from NY
.setQuery(filteredQuery(termQuery("mappedfsn",mapped), geoBoundingBoxFilter("point").cache(true).topLeft(latt+ spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt- spreadOnLatitude, longii-spreadOnLongitude)))
.setSize(NumRes).execute().actionGet();

This is how i am searching. Where i m wrong in this.
Regards
Prashant

On Wed, Feb 1, 2012 at 3:13 PM, Shay Banon <kimchy@gmail.com (mailto:kimchy@gmail.com)> wrote:

Whats was not lucky? What did not work?

On Wednesday, February 1, 2012 at 6:31 AM, BeyondLimit wrote:

hi Shay,
i guess you are talking about like this
String mapping =
XContentFactory.jsonBuilder().startObject().startObject("type")
.startObject("properties").startObject("point").field("type",
"geo_point").field("lat_lon", true).endObject().endObject()
.endObject().endObject().string();
But then also no luck. Can anybody help me out. Explaining better way
of using BoundaryBox search in java.
Regards
Prashant

On Jan 31, 10:33 pm, BeyondLimit <prashant.vi...@gmail.com (http://gmail.com)> wrote:

Hi Thanks for the reply.
But can you share something using java , coz i am new to CURL .
It will be great if u can explain i java language.
Any java example which explains what you are saying.
Regards
Prashant
On Jan 31, 10:27 pm, Shay Banon <kim...@gmail.com (http://gmail.com)> wrote:

Also note, that having the index in ram drive will not matter that much, since by default the bbox checks are done in memory with values loaded. For indexed type execution, it will help.

On Tuesday, January 31, 2012 at 7:26 PM, Shay Banon wrote:

You can try and use different execution mode (documented under type at the bottom herehttp://www.elasticsearch.org/guide/reference/query-dsl/geo-bounding-b...) to have it indexed instead of done in memory. As explained there, you will need to have the geo_point type mapping index the lat/lon as well in this case.

On Tuesday, January 31, 2012 at 6:39 PM, BeyondLimit wrote:

Hi there ,
Actually i have just stated using Elastic Search , and i have a
situation where in i have to search in the Boundary of lat/long .
When i have created the boundary and performed the search using java
code given below.

client.prepareSearch() .setQuery(filteredQuery(textQuery("mappedfsn",
mapped), geoBoundingBoxFilter("location").topLeft(latt+
spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt-
spreadOnLatitude, longii-
spreadOnLongitude))).setSize(NumRes).execute().actionGet();

I found the search taking too much of time (200 mili sec on the Big
Server with 24gb of ram on it and i kept index on the Ram Disk too)
Now if i remove the boundingbox condition i get the result in no time
say 20 mili sec.

Can any body help me on this.

Thanks in advance.
Regards
Prashant


(prashant5375) #10

Hi Shay
I was doing hit and try thats why you see cache = true but i have
removed that too , and used type("indexed") now its more slow.
And if i remove type("indexed") its slow too.

I have been following the Examples mention in the ElasticSearch (in
github "GeoBoundingBoxTests.java").
What i am missing in it. I am badly needed this thing to be working.

Earlier i used to search Using Lucene Api and search on Numeric Range
Query and it was fast.But there was a problem that the retrieval of
Document was slow.
So on the production server i was getting response time of 80 milisec.

But after reading lots of Search i cam to know about Elastic Search.
Here the retrieval is fast but the Query time is much higher than
what i was getting on my implementation of Lucene.

If you can or any one can help me to resolve the problem

Regards
Prashant

On Feb 1, 4:12 pm, Shay Banon kim...@gmail.com wrote:

Few problems in your code:

  1. Don't set cache to true, as its meaningless when using something like bounding box since it changes all the time. If you set cache to true, it will need to go over all the docs in order to create a matching doc set for them.

  2. In order to check if the indexed option is faster than the memory one, you need to call type("indexed") on the geo bounding box (or type("memory") which is the default).

On Wednesday, February 1, 2012 at 12:09 PM, PS wrote:

Hi ,
I am writing the problem again :
I have a big server of 25 gb ram on it.
And i have a index of 13 gb . When i am running only textQuery it giving me results very fast say 10 mili sec.
but when i an using boundary box its taking 200+ mili sec in query execution.

How i have created the Index given below.
------Indexing------------
/*This is the mapping i have created
*/
String mapping = XContentFactory.jsonBuilder().startObject().startObject("type")
.startObject("properties").startObject("point").field("type", "geo_point").field("lat_lon", true).endObject().endObject()
.endObject().endObject().string();

//And this is who i am indexing it

brb.add(client.prepareIndex("test", "type", j+"").setSource(jsonBuilder().startObject()
.field("recid", tmp[1])
.field("mergefield", tmp[4])
.field("mappedfsn", tmp[0].replaceAll(""", ""))
.startObject("point").field("lat", Double.parseDouble(lati)).field("lon", Double.parseDouble(longi)).endObject()
.endObject())
);

------------Search------------
response = client.prepareSearch() // from NY
.setQuery(filteredQuery(termQuery("mappedfsn",mapped), geoBoundingBoxFilter("point").cache(true).topLeft(latt+ spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt- spreadOnLatitude, longii-spreadOnLongitude)))
.setSize(NumRes).execute().actionGet();

This is how i am searching. Where i m wrong in this.
Regards
Prashant

On Wed, Feb 1, 2012 at 3:13 PM, Shay Banon <kim...@gmail.com (mailto:kim...@gmail.com)> wrote:

Whats was not lucky? What did not work?

On Wednesday, February 1, 2012 at 6:31 AM, BeyondLimit wrote:

hi Shay,
i guess you are talking about like this
String mapping =
XContentFactory.jsonBuilder().startObject().startObject("type")
.startObject("properties").startObject("point").field("type",
"geo_point").field("lat_lon", true).endObject().endObject()
.endObject().endObject().string();
But then also no luck. Can anybody help me out. Explaining better way
of using BoundaryBox search in java.
Regards
Prashant

On Jan 31, 10:33 pm, BeyondLimit <prashant.vi...@gmail.com (http://gmail.com)> wrote:

Hi Thanks for the reply.
But can you share something using java , coz i am new to CURL .
It will be great if u can explain i java language.
Any java example which explains what you are saying.
Regards
Prashant
On Jan 31, 10:27 pm, Shay Banon <kim...@gmail.com (http://gmail.com)> wrote:

Also note, that having the index in ram drive will not matter that much, since by default the bbox checks are done in memory with values loaded. For indexed type execution, it will help.

On Tuesday, January 31, 2012 at 7:26 PM, Shay Banon wrote:

You can try and use different execution mode (documented under type at the bottom herehttp://www.elasticsearch.org/guide/reference/query-dsl/geo-bounding-b...) to have it indexed instead of done in memory. As explained there, you will need to have the geo_point type mapping index the lat/lon as well in this case.

On Tuesday, January 31, 2012 at 6:39 PM, BeyondLimit wrote:

Hi there ,
Actually i have just stated using Elastic Search , and i have a
situation where in i have to search in the Boundary of lat/long .
When i have created the boundary and performed the search using java
code given below.

client.prepareSearch() .setQuery(filteredQuery(textQuery("mappedfsn",
mapped), geoBoundingBoxFilter("location").topLeft(latt+
spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt-
spreadOnLatitude, longii-
spreadOnLongitude))).setSize(NumRes).execute().actionGet();

I found the search taking too much of time (200 mili sec on the Big
Server with 24gb of ram on it and i kept index on the Ram Disk too)
Now if i remove the boundingbox condition i get the result in no time
say 20 mili sec.

Can any body help me on this.

Thanks in advance.
Regards
Prashant


(prashant5375) #11

Hi Shay ,
I know you are very busy giving answer to others query .
Can you please spare some time to answer my question i have asked in
the mail chain given below.
And also if there is any limitation of elastic search , that too tell
me. I know there must be some workaround of this problem,
and only you can answer the question
Regards
Prashant

On Feb 1, 7:30 pm, BeyondLimit prashant.vi...@gmail.com wrote:

Hi Shay
I was doing hit and try thats why you see cache = true but i have
removed that too , and used type("indexed") now its more slow.
And if i remove type("indexed") its slow too.

I have been following the Examples mention in the ElasticSearch (in
github "GeoBoundingBoxTests.java").
What i am missing in it. I am badly needed this thing to be working.

Earlier i used to search Using Lucene Api and search on Numeric Range
Query and it was fast.But there was a problem that the retrieval of
Document was slow.
So on the production server i was getting response time of 80 milisec.

But after reading lots of Search i cam to know about Elastic Search.
Here the retrieval is fast but the Query time is much higher than
what i was getting on my implementation of Lucene.

If you can or any one can help me to resolve the problem

Regards
Prashant

On Feb 1, 4:12 pm, Shay Banon kim...@gmail.com wrote:

Few problems in your code:

  1. Don't set cache to true, as its meaningless when using something like bounding box since it changes all the time. If you set cache to true, it will need to go over all the docs in order to create a matching doc set for them.
  1. In order to check if the indexed option is faster than the memory one, you need to call type("indexed") on the geo bounding box (or type("memory") which is the default).

On Wednesday, February 1, 2012 at 12:09 PM, PS wrote:

Hi ,
I am writing the problem again :
I have a big server of 25 gb ram on it.
And i have a index of 13 gb . When i am running only textQuery it giving me results very fast say 10 mili sec.
but when i an using boundary box its taking 200+ mili sec in query execution.

How i have created the Index given below.
------Indexing------------
/*This is the mapping i have created
*/
String mapping = XContentFactory.jsonBuilder().startObject().startObject("type")
.startObject("properties").startObject("point").field("type", "geo_point").field("lat_lon", true).endObject().endObject()
.endObject().endObject().string();

//And this is who i am indexing it

brb.add(client.prepareIndex("test", "type", j+"").setSource(jsonBuilder().startObject()
.field("recid", tmp[1])
.field("mergefield", tmp[4])
.field("mappedfsn", tmp[0].replaceAll(""", ""))
.startObject("point").field("lat", Double.parseDouble(lati)).field("lon", Double.parseDouble(longi)).endObject()
.endObject())
);

------------Search------------
response = client.prepareSearch() // from NY
.setQuery(filteredQuery(termQuery("mappedfsn",mapped), geoBoundingBoxFilter("point").cache(true).topLeft(latt+ spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt- spreadOnLatitude, longii-spreadOnLongitude)))
.setSize(NumRes).execute().actionGet();

This is how i am searching. Where i m wrong in this.
Regards
Prashant

On Wed, Feb 1, 2012 at 3:13 PM, Shay Banon <kim...@gmail.com (mailto:kim...@gmail.com)> wrote:

Whats was not lucky? What did not work?

On Wednesday, February 1, 2012 at 6:31 AM, BeyondLimit wrote:

hi Shay,
i guess you are talking about like this
String mapping =
XContentFactory.jsonBuilder().startObject().startObject("type")
.startObject("properties").startObject("point").field("type",
"geo_point").field("lat_lon", true).endObject().endObject()
.endObject().endObject().string();
But then also no luck. Can anybody help me out. Explaining better way
of using BoundaryBox search in java.
Regards
Prashant

On Jan 31, 10:33 pm, BeyondLimit <prashant.vi...@gmail.com (http://gmail.com)> wrote:

Hi Thanks for the reply.
But can you share something using java , coz i am new to CURL .
It will be great if u can explain i java language.
Any java example which explains what you are saying.
Regards
Prashant
On Jan 31, 10:27 pm, Shay Banon <kim...@gmail.com (http://gmail.com)> wrote:

Also note, that having the index in ram drive will not matter that much, since by default the bbox checks are done in memory with values loaded. For indexed type execution, it will help.

On Tuesday, January 31, 2012 at 7:26 PM, Shay Banon wrote:

You can try and use different execution mode (documented under type at the bottom herehttp://www.elasticsearch.org/guide/reference/query-dsl/geo-bounding-b...) to have it indexed instead of done in memory. As explained there, you will need to have the geo_point type mapping index the lat/lon as well in this case.

On Tuesday, January 31, 2012 at 6:39 PM, BeyondLimit wrote:

Hi there ,
Actually i have just stated using Elastic Search , and i have a
situation where in i have to search in the Boundary of lat/long .
When i have created the boundary and performed the search using java
code given below.

client.prepareSearch() .setQuery(filteredQuery(textQuery("mappedfsn",
mapped), geoBoundingBoxFilter("location").topLeft(latt+
spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt-
spreadOnLatitude, longii-
spreadOnLongitude))).setSize(NumRes).execute().actionGet();

I found the search taking too much of time (200 mili sec on the Big
Server with 24gb of ram on it and i kept index on the Ram Disk too)
Now if i remove the boundingbox condition i get the result in no time
say 20 mili sec.

Can any body help me on this.

Thanks in advance.
Regards
Prashant


(Dennis) #12

I'm starting a project that will use geoSearch. If this is the speed,
that ES does, it would be a deal killer. I would have 10's of
thousands of users accessing geoSearched items all the time. I'm
looking to see how this important issue is resolved.

On Feb 1, 9:00 pm, BeyondLimit prashant.vi...@gmail.com wrote:

Hi Shay ,
I know you are very busy giving answer to others query .
Can you please spare some time to answer my question i have asked in
the mail chain given below.
And also if there is any limitation of elastic search , that too tell
me. I know there must be some workaround of this problem,
and only you can answer the question
Regards
Prashant

On Feb 1, 7:30 pm, BeyondLimit prashant.vi...@gmail.com wrote:

Hi Shay
I was doing hit and try thats why you see cache = true but i have
removed that too , and used type("indexed") now its more slow.
And if i remove type("indexed") its slow too.

I have been following the Examples mention in the ElasticSearch (in
github "GeoBoundingBoxTests.java").
What i am missing in it. I am badly needed this thing to be working.

Earlier i used to search Using Lucene Api and search on Numeric Range
Query and it was fast.But there was a problem that the retrieval of
Document was slow.
So on the production server i was getting response time of 80 milisec.

But after reading lots of Search i cam to know about Elastic Search.
Here the retrieval is fast but the Query time is much higher than
what i was getting on my implementation of Lucene.

If you can or any one can help me to resolve the problem

Regards
Prashant

On Feb 1, 4:12 pm, Shay Banon kim...@gmail.com wrote:

Few problems in your code:

  1. Don't set cache to true, as its meaningless when using something like bounding box since it changes all the time. If you set cache to true, it will need to go over all the docs in order to create a matching doc set for them.
  1. In order to check if the indexed option is faster than the memory one, you need to call type("indexed") on the geo bounding box (or type("memory") which is the default).

On Wednesday, February 1, 2012 at 12:09 PM, PS wrote:

Hi ,
I am writing the problem again :
I have a big server of 25 gb ram on it.
And i have a index of 13 gb . When i am running only textQuery it giving me results very fast say 10 mili sec.
but when i an using boundary box its taking 200+ mili sec in query execution.

How i have created the Index given below.
------Indexing------------
/*This is the mapping i have created
*/
String mapping = XContentFactory.jsonBuilder().startObject().startObject("type")
.startObject("properties").startObject("point").field("type", "geo_point").field("lat_lon", true).endObject().endObject()
.endObject().endObject().string();

//And this is who i am indexing it

brb.add(client.prepareIndex("test", "type", j+"").setSource(jsonBuilder().startObject()
.field("recid", tmp[1])
.field("mergefield", tmp[4])
.field("mappedfsn", tmp[0].replaceAll(""", ""))
.startObject("point").field("lat", Double.parseDouble(lati)).field("lon", Double.parseDouble(longi)).endObject()
.endObject())
);

------------Search------------
response = client.prepareSearch() // from NY
.setQuery(filteredQuery(termQuery("mappedfsn",mapped), geoBoundingBoxFilter("point").cache(true).topLeft(latt+ spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt- spreadOnLatitude, longii-spreadOnLongitude)))
.setSize(NumRes).execute().actionGet();

This is how i am searching. Where i m wrong in this.
Regards
Prashant

On Wed, Feb 1, 2012 at 3:13 PM, Shay Banon <kim...@gmail.com (mailto:kim...@gmail.com)> wrote:

Whats was not lucky? What did not work?

On Wednesday, February 1, 2012 at 6:31 AM, BeyondLimit wrote:

hi Shay,
i guess you are talking about like this
String mapping =
XContentFactory.jsonBuilder().startObject().startObject("type")
.startObject("properties").startObject("point").field("type",
"geo_point").field("lat_lon", true).endObject().endObject()
.endObject().endObject().string();
But then also no luck. Can anybody help me out. Explaining better way
of using BoundaryBox search in java.
Regards
Prashant

On Jan 31, 10:33 pm, BeyondLimit <prashant.vi...@gmail.com (http://gmail.com)> wrote:

Hi Thanks for the reply.
But can you share something using java , coz i am new to CURL .
It will be great if u can explain i java language.
Any java example which explains what you are saying.
Regards
Prashant
On Jan 31, 10:27 pm, Shay Banon <kim...@gmail.com (http://gmail.com)> wrote:

Also note, that having the index in ram drive will not matter that much, since by default the bbox checks are done in memory with values loaded. For indexed type execution, it will help.

On Tuesday, January 31, 2012 at 7:26 PM, Shay Banon wrote:

You can try and use different execution mode (documented under type at the bottom herehttp://www.elasticsearch.org/guide/reference/query-dsl/geo-bounding-b...) to have it indexed instead of done in memory. As explained there, you will need to have the geo_point type mapping index the lat/lon as well in this case.

On Tuesday, January 31, 2012 at 6:39 PM, BeyondLimit wrote:

Hi there ,
Actually i have just stated using Elastic Search , and i have a
situation where in i have to search in the Boundary of lat/long .
When i have created the boundary and performed the search using java
code given below.

client.prepareSearch() .setQuery(filteredQuery(textQuery("mappedfsn",
mapped), geoBoundingBoxFilter("location").topLeft(latt+
spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt-
spreadOnLatitude, longii-
spreadOnLongitude))).setSize(NumRes).execute().actionGet();

I found the search taking too much of time (200 mili sec on the Big
Server with 24gb of ram on it and i kept index on the Ram Disk too)
Now if i remove the boundingbox condition i get the result in no time
say 20 mili sec.

Can any body help me on this.

Thanks in advance.
Regards
Prashant


(prashant5375) #13

Hi Shay,
I have been seen you are replying to the queries of other.
Please let me know when can i expect the answer of my query.
Regards
Prashant

On Feb 2, 10:00 am, BeyondLimit prashant.vi...@gmail.com wrote:

Hi Shay ,
I know you are very busy giving answer to others query .
Can you please spare some time to answer my question i have asked in
the mail chain given below.
And also if there is any limitation of elastic search , that too tell
me. I know there must be some workaround of this problem,
and only you can answer the question
Regards
Prashant

On Feb 1, 7:30 pm, BeyondLimit prashant.vi...@gmail.com wrote:

Hi Shay
I was doing hit and try thats why you see cache = true but i have
removed that too , and used type("indexed") now its more slow.
And if i remove type("indexed") its slow too.

I have been following the Examples mention in the ElasticSearch (in
github "GeoBoundingBoxTests.java").
What i am missing in it. I am badly needed this thing to be working.

Earlier i used to search Using Lucene Api and search on Numeric Range
Query and it was fast.But there was a problem that the retrieval of
Document was slow.
So on the production server i was getting response time of 80 milisec.

But after reading lots of Search i cam to know about Elastic Search.
Here the retrieval is fast but the Query time is much higher than
what i was getting on my implementation of Lucene.

If you can or any one can help me to resolve the problem

Regards
Prashant

On Feb 1, 4:12 pm, Shay Banon kim...@gmail.com wrote:

Few problems in your code:

  1. Don't set cache to true, as its meaningless when using something like bounding box since it changes all the time. If you set cache to true, it will need to go over all the docs in order to create a matching doc set for them.
  1. In order to check if the indexed option is faster than the memory one, you need to call type("indexed") on the geo bounding box (or type("memory") which is the default).

On Wednesday, February 1, 2012 at 12:09 PM, PS wrote:

Hi ,
I am writing the problem again :
I have a big server of 25 gb ram on it.
And i have a index of 13 gb . When i am running only textQuery it giving me results very fast say 10 mili sec.
but when i an using boundary box its taking 200+ mili sec in query execution.

How i have created the Index given below.
------Indexing------------
/*This is the mapping i have created
*/
String mapping = XContentFactory.jsonBuilder().startObject().startObject("type")
.startObject("properties").startObject("point").field("type", "geo_point").field("lat_lon", true).endObject().endObject()
.endObject().endObject().string();

//And this is who i am indexing it

brb.add(client.prepareIndex("test", "type", j+"").setSource(jsonBuilder().startObject()
.field("recid", tmp[1])
.field("mergefield", tmp[4])
.field("mappedfsn", tmp[0].replaceAll(""", ""))
.startObject("point").field("lat", Double.parseDouble(lati)).field("lon", Double.parseDouble(longi)).endObject()
.endObject())
);

------------Search------------
response = client.prepareSearch() // from NY
.setQuery(filteredQuery(termQuery("mappedfsn",mapped), geoBoundingBoxFilter("point").cache(true).topLeft(latt+ spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt- spreadOnLatitude, longii-spreadOnLongitude)))
.setSize(NumRes).execute().actionGet();

This is how i am searching. Where i m wrong in this.
Regards
Prashant

On Wed, Feb 1, 2012 at 3:13 PM, Shay Banon <kim...@gmail.com (mailto:kim...@gmail.com)> wrote:

Whats was not lucky? What did not work?

On Wednesday, February 1, 2012 at 6:31 AM, BeyondLimit wrote:

hi Shay,
i guess you are talking about like this
String mapping =
XContentFactory.jsonBuilder().startObject().startObject("type")
.startObject("properties").startObject("point").field("type",
"geo_point").field("lat_lon", true).endObject().endObject()
.endObject().endObject().string();
But then also no luck. Can anybody help me out. Explaining better way
of using BoundaryBox search in java.
Regards
Prashant

On Jan 31, 10:33 pm, BeyondLimit <prashant.vi...@gmail.com (http://gmail.com)> wrote:

Hi Thanks for the reply.
But can you share something using java , coz i am new to CURL .
It will be great if u can explain i java language.
Any java example which explains what you are saying.
Regards
Prashant
On Jan 31, 10:27 pm, Shay Banon <kim...@gmail.com (http://gmail.com)> wrote:

Also note, that having the index in ram drive will not matter that much, since by default the bbox checks are done in memory with values loaded. For indexed type execution, it will help.

On Tuesday, January 31, 2012 at 7:26 PM, Shay Banon wrote:

You can try and use different execution mode (documented under type at the bottom herehttp://www.elasticsearch.org/guide/reference/query-dsl/geo-bounding-b...) to have it indexed instead of done in memory. As explained there, you will need to have the geo_point type mapping index the lat/lon as well in this case.

On Tuesday, January 31, 2012 at 6:39 PM, BeyondLimit wrote:

Hi there ,
Actually i have just stated using Elastic Search , and i have a
situation where in i have to search in the Boundary of lat/long .
When i have created the boundary and performed the search using java
code given below.

client.prepareSearch() .setQuery(filteredQuery(textQuery("mappedfsn",
mapped), geoBoundingBoxFilter("location").topLeft(latt+
spreadOnLatitude, longii+ spreadOnLongitude).bottomRight(latt-
spreadOnLatitude, longii-
spreadOnLongitude))).setSize(NumRes).execute().actionGet();

I found the search taking too much of time (200 mili sec on the Big
Server with 24gb of ram on it and i kept index on the Ram Disk too)
Now if i remove the boundingbox condition i get the result in no time
say 20 mili sec.

Can any body help me on this.

Thanks in advance.
Regards
Prashant


(gearond) #14

did you get this resolved? I used solr with a plugin that pre indexed the geocode into a special format, and it was blindingly fast. I think that they were suggesting somethng like that. did you do that kind of indexing?


(prashant5375) #15

Hi there ,
No i did not get that resolved.
I did not understand what are you talking about.
Can you please explain bit more so that i can follow the steps .
Thanks in advance.
_Prashant

On Sun, Mar 4, 2012 at 10:31 PM, gearond gearond@sbcglobal.net wrote:

did you get this resolved? I used solr with a plugin that pre indexed the
geocode into a special format, and it was blindingly fast. I think that
they
were suggesting somethng like that. did you do that kind of indexing?

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/BoundingBox-Search-takes-time-tp3703932p3798475.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.


(system) #16