Query never returns


(Matt Weber) #1

I have a need to make sure all hits from a specific field in a specific type are always the highest ranked documents. To do this, I have used a constant score query with a really high boost value against this type. At the same time, I still want to search across other types and fields in my index. I have crafted a boolean query with 2 should clauses. The first contains the constant score query and the second contains a query string query. Everything works as expected until my constant score query uses a filter. When I specify a filter, the query locks up and never returns a response.

The only reason I decided to use a filter was for better performance. If I remove the 2nd clause from the boolean query, the search executes just fine. If I leave the 2nd clause, and remove the filter from the constant score query, it also returns fine. If I do the constant score filter query by itself, it also returns the expected query.

Here is a gist with the simplest example of the problem:

I am testing with 0.19.0.RC2.

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


(Shay Banon) #2

Regarding the recreation, I ran it against a simple index, and it doesn't lock up, can you provide a recreation that shows the lock up starting fresh (possibly indexing sample docs)?

Regarding boosting specific type, you can use the custom filters score query (http://www.elasticsearch.org/guide/reference/query-dsl/custom-filters-score-query.html), and have a term filter on _type that boosts the score if the document matches that type.

On Saturday, February 18, 2012 at 8:43 PM, Matt Weber wrote:

I have a need to make sure all hits from a specific field in a specific type are always the highest ranked documents. To do this, I have used a constant score query with a really high boost value against this type. At the same time, I still want to search across other types and fields in my index. I have crafted a boolean query with 2 should clauses. The first contains the constant score query and the second contains a query string query. Everything works as expected until my constant score query uses a filter. When I specify a filter, the query locks up and never returns a response.

The only reason I decided to use a filter was for better performance. If I remove the 2nd clause from the boolean query, the search executes just fine. If I leave the 2nd clause, and remove the filter from the constant score query, it also returns fine. If I do the constant score filter query by itself, it also returns the expected query.

Here is a gist with the simplest example of the problem:
https://gist.github.com/1860568

I am testing with 0.19.0.RC2.

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


(Matt Weber) #3

The gist has the sample docs and sample query that locks it up. Just to double check I installed a fresh copy 0.19.0.RC2, started it, copy and pasted the test docs from the gist, then copy and pasted the query from the gist and it locked up.

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

On Monday, February 20, 2012 at 5:05 AM, Shay Banon wrote:

Regarding the recreation, I ran it against a simple index, and it doesn't lock up, can you provide a recreation that shows the lock up starting fresh (possibly indexing sample docs)?

Regarding boosting specific type, you can use the custom filters score query (http://www.elasticsearch.org/guide/reference/query-dsl/custom-filters-score-query.html), and have a term filter on _type that boosts the score if the document matches that type.

On Saturday, February 18, 2012 at 8:43 PM, Matt Weber wrote:

I have a need to make sure all hits from a specific field in a specific type are always the highest ranked documents. To do this, I have used a constant score query with a really high boost value against this type. At the same time, I still want to search across other types and fields in my index. I have crafted a boolean query with 2 should clauses. The first contains the constant score query and the second contains a query string query. Everything works as expected until my constant score query uses a filter. When I specify a filter, the query locks up and never returns a response.

The only reason I decided to use a filter was for better performance. If I remove the 2nd clause from the boolean query, the search executes just fine. If I leave the 2nd clause, and remove the filter from the constant score query, it also returns fine. If I do the constant score filter query by itself, it also returns the expected query.

Here is a gist with the simplest example of the problem:
https://gist.github.com/1860568

I am testing with 0.19.0.RC2.

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


(egaumer) #4

I was able to reproduce this based on the gist. Debug logs don't show
anything but the CPU spikes (when you run the query) and never comes back
down (even when you kill the query). You actually have to restart
elasticsearch.


(Shay Banon) #5

It seems like its stuck in a look within Lucene boolean query, still need to chase it more and figure out what goes wrong, can you open an issue just so we won't loose it?

On Monday, February 20, 2012 at 5:19 PM, Matt Weber wrote:

The gist has the sample docs and sample query that locks it up. Just to double check I installed a fresh copy 0.19.0.RC2, started it, copy and pasted the test docs from the gist, then copy and pasted the query from the gist and it locked up.

https://gist.github.com/1860568

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

On Monday, February 20, 2012 at 5:05 AM, Shay Banon wrote:

Regarding the recreation, I ran it against a simple index, and it doesn't lock up, can you provide a recreation that shows the lock up starting fresh (possibly indexing sample docs)?

Regarding boosting specific type, you can use the custom filters score query (http://www.elasticsearch.org/guide/reference/query-dsl/custom-filters-score-query.html), and have a term filter on _type that boosts the score if the document matches that type.

On Saturday, February 18, 2012 at 8:43 PM, Matt Weber wrote:

I have a need to make sure all hits from a specific field in a specific type are always the highest ranked documents. To do this, I have used a constant score query with a really high boost value against this type. At the same time, I still want to search across other types and fields in my index. I have crafted a boolean query with 2 should clauses. The first contains the constant score query and the second contains a query string query. Everything works as expected until my constant score query uses a filter. When I specify a filter, the query locks up and never returns a response.

The only reason I decided to use a filter was for better performance. If I remove the 2nd clause from the boolean query, the search executes just fine. If I leave the 2nd clause, and remove the filter from the constant score query, it also returns fine. If I do the constant score filter query by itself, it also returns the expected query.

Here is a gist with the simplest example of the problem:
https://gist.github.com/1860568

I am testing with 0.19.0.RC2.

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


(Ivan Brusic) #6

I have seen queries not returning if there are older versions of
Lucene in the classpath. Not every query uses every Lucene
class/method. Certain queries might use a different method, and if the
previous version of Lucene does not have that method, a query to ES
will simply hang with no exception thrown on the client or server
side.

--
Ivan

On Tue, Feb 21, 2012 at 8:29 AM, Shay Banon kimchy@gmail.com wrote:

It seems like its stuck in a look within Lucene boolean query, still need to
chase it more and figure out what goes wrong, can you open an issue just so
we won't loose it?

On Monday, February 20, 2012 at 5:19 PM, Matt Weber wrote:

The gist has the sample docs and sample query that locks it up. Just to
double check I installed a fresh copy 0.19.0.RC2, started it, copy and
pasted the test docs from the gist, then copy and pasted the query from the
gist and it locked up.

https://gist.github.com/1860568

--
Matt Weber
Sent with Sparrow

On Monday, February 20, 2012 at 5:05 AM, Shay Banon wrote:

Regarding the recreation, I ran it against a simple index, and it doesn't
lock up, can you provide a recreation that shows the lock up starting fresh
(possibly indexing sample docs)?

Regarding boosting specific type, you can use the custom filters score query
(http://www.elasticsearch.org/guide/reference/query-dsl/custom-filters-score-query.html),
and have a term filter on _type that boosts the score if the document
matches that type.

On Saturday, February 18, 2012 at 8:43 PM, Matt Weber wrote:

I have a need to make sure all hits from a specific field in a specific type
are always the highest ranked documents. To do this, I have used a constant
score query with a really high boost value against this type. At the same
time, I still want to search across other types and fields in my index. I
have crafted a boolean query with 2 should clauses. The first contains the
constant score query and the second contains a query string query.
Everything works as expected until my constant score query uses a filter.
When I specify a filter, the query locks up and never returns a response.

The only reason I decided to use a filter was for better performance. If I
remove the 2nd clause from the boolean query, the search executes just fine.
If I leave the 2nd clause, and remove the filter from the constant score
query, it also returns fine. If I do the constant score filter query by
itself, it also returns the expected query.

Here is a gist with the simplest example of the problem:
https://gist.github.com/1860568

I am testing with 0.19.0.RC2.

--
Matt Weber
Sent with Sparrow


(Matt Weber) #7

Thanks!

--
Matt Weber
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

On Tuesday, February 21, 2012 at 9:17 AM, Ivan Brusic wrote:

I have seen queries not returning if there are older versions of
Lucene in the classpath. Not every query uses every Lucene
class/method. Certain queries might use a different method, and if the
previous version of Lucene does not have that method, a query to ES
will simply hang with no exception thrown on the client or server
side.

--
Ivan

On Tue, Feb 21, 2012 at 8:29 AM, Shay Banon <kimchy@gmail.com (mailto:kimchy@gmail.com)> wrote:

It seems like its stuck in a look within Lucene boolean query, still need to
chase it more and figure out what goes wrong, can you open an issue just so
we won't loose it?

On Monday, February 20, 2012 at 5:19 PM, Matt Weber wrote:

The gist has the sample docs and sample query that locks it up. Just to
double check I installed a fresh copy 0.19.0.RC2, started it, copy and
pasted the test docs from the gist, then copy and pasted the query from the
gist and it locked up.

https://gist.github.com/1860568

--
Matt Weber
Sent with Sparrow

On Monday, February 20, 2012 at 5:05 AM, Shay Banon wrote:

Regarding the recreation, I ran it against a simple index, and it doesn't
lock up, can you provide a recreation that shows the lock up starting fresh
(possibly indexing sample docs)?

Regarding boosting specific type, you can use the custom filters score query
(http://www.elasticsearch.org/guide/reference/query-dsl/custom-filters-score-query.html),
and have a term filter on _type that boosts the score if the document
matches that type.

On Saturday, February 18, 2012 at 8:43 PM, Matt Weber wrote:

I have a need to make sure all hits from a specific field in a specific type
are always the highest ranked documents. To do this, I have used a constant
score query with a really high boost value against this type. At the same
time, I still want to search across other types and fields in my index. I
have crafted a boolean query with 2 should clauses. The first contains the
constant score query and the second contains a query string query.
Everything works as expected until my constant score query uses a filter.
When I specify a filter, the query locks up and never returns a response.

The only reason I decided to use a filter was for better performance. If I
remove the 2nd clause from the boolean query, the search executes just fine.
If I leave the 2nd clause, and remove the filter from the constant score
query, it also returns fine. If I do the constant score filter query by
itself, it also returns the expected query.

Here is a gist with the simplest example of the problem:
https://gist.github.com/1860568

I am testing with 0.19.0.RC2.

--
Matt Weber
Sent with Sparrow


(system) #8