Major performance degradation on 0.12 for queries like *:* AND NOT <queryterm>


(ppearcy) #1

Hi,
I am seeing some really nasty performance with negative queries on
0.12. I know these queries performed fine on a pre 0.12 release (maybe
0.10 or 0.11). To test that it wasn't something on my side, I created
th sample twitter index and add a couple of test docs (basically, what
is in the REST docs).

Then I perform query like this:
http://localhost:9200/twitter/_search?q=:%20AND%20NOT%20elastic

Where the non-URL encoded query is:
: AND NOT elastic

The query takes a very long time and pegs a CPU while it is running.

I can open a ticket to track further, if needed or verify any updates.

Thanks!
Paul


(ppearcy) #2

FYI, playing around more, the performance degradation seems to be
related to the extra AND : I am adding to the query that ES does not
need (but pure Lucene query syntax does). So, I have an update (remove
extraneous :) on my side that should hopefully address.

Thanks

On Oct 25, 4:33 pm, Paul ppea...@gmail.com wrote:

Hi,
I am seeing some really nasty performance with negative queries on
0.12. I know these queries performed fine on a pre 0.12 release (maybe
0.10 or 0.11). To test that it wasn't something on my side, I created
th sample twitter index and add a couple of test docs (basically, what
is in the REST docs).

Then I perform query like this:http://localhost:9200/twitter/_search?q=:%20AND%20NOT%20elastic

Where the non-URL encoded query is:
: AND NOT elastic

The query takes a very long time and pegs a CPU while it is running.

I can open a ticket to track further, if needed or verify any updates.

Thanks!
Paul


(ppearcy) #3

Further update, by removing the unseeded : I have confirmed that
search times are blazing again.

On Oct 25, 4:42 pm, Paul ppea...@gmail.com wrote:

FYI, playing around more, the performance degradation seems to be
related to the extra AND : I am adding to the query that ES does not
need (but pure Lucene query syntax does). So, I have an update (remove
extraneous :) on my side that should hopefully address.

Thanks

On Oct 25, 4:33 pm, Paul ppea...@gmail.com wrote:

Hi,
I am seeing some really nasty performance with negative queries on
0.12. I know these queries performed fine on a pre 0.12 release (maybe
0.10 or 0.11). To test that it wasn't something on my side, I created
th sample twitter index and add a couple of test docs (basically, what
is in the REST docs).

Then I perform query like this:http://localhost:9200/twitter/_search?q=:%20AND%20NOT%20elastic

Where the non-URL encoded query is:
: AND NOT elastic

The query takes a very long time and pegs a CPU while it is running.

I can open a ticket to track further, if needed or verify any updates.

Thanks!
Paul


(Clinton Gormley) #4

On Mon, 2010-10-25 at 16:58 -0700, Paul wrote:

Further update, by removing the unseeded : I have confirmed that
search times are blazing again.

Paul - please keep an eye out for other slowdowns - I am seeing some
very slow responses on 0.12 for queries which were working well before.

clint

--
Web Announcements Limited is a company registered in England and Wales,
with company number 05608868, with registered address at 10 Arvon Road,
London, N5 1PR.


(ppearcy) #5

Hey Clint,
I can confirm that this is the only slow down we have run into. I've
pumped millions of real world queries through since, mostly consisting
of complex nested fielded boolean queries sorted by date and there are
no other performance differences. I'm definitely only touching a
fraction of the search functionality, though.

Thanks,
Paul

On Oct 25, 10:51 pm, Clinton Gormley clin...@iannounce.co.uk wrote:

On Mon, 2010-10-25 at 16:58 -0700, Paul wrote:

Further update, by removing the unseeded : I have confirmed that
search times are blazing again.

Paul - please keep an eye out for other slowdowns - I am seeing some
very slow responses on 0.12 for queries which were working well before.

clint

--
Web Announcements Limited is a company registered in England and Wales,
with company number 05608868, with registered address at 10 Arvon Road,
London, N5 1PR.


(Shay Banon) #6

Hi,

I added an optimization in match_all cases (including query string like
:), but, that optimization has resulted in excessive processing in some
cases. Easily fixable while still maintaing the optimization. I have created
a branch for 0.12 on github with the fix:
http://github.com/elasticsearch/elasticsearch/tree/0.12. I will release
0.12.1 pretty soon (likely today) as this is a major regression. Can you
give it a go?

-shay.banon

On Tue, Oct 26, 2010 at 9:47 AM, Paul ppearcy@gmail.com wrote:

Hey Clint,
I can confirm that this is the only slow down we have run into. I've
pumped millions of real world queries through since, mostly consisting
of complex nested fielded boolean queries sorted by date and there are
no other performance differences. I'm definitely only touching a
fraction of the search functionality, though.

Thanks,
Paul

On Oct 25, 10:51 pm, Clinton Gormley clin...@iannounce.co.uk wrote:

On Mon, 2010-10-25 at 16:58 -0700, Paul wrote:

Further update, by removing the unseeded : I have confirmed that
search times are blazing again.

Paul - please keep an eye out for other slowdowns - I am seeing some
very slow responses on 0.12 for queries which were working well before.

clint

--
Web Announcements Limited is a company registered in England and Wales,
with company number 05608868, with registered address at 10 Arvon Road,
London, N5 1PR.


(Clinton Gormley) #7

Hi kimchy

I added an optimization in match_all cases (including query string
like :), but, that optimization has resulted in excessive processing
in some cases. Easily fixable while still maintaing the optimization.
I have created a branch for 0.12 on github with the fix:
http://github.com/elasticsearch/elasticsearch/tree/0.12. I will
release 0.12.1 pretty soon (likely today) as this is a major
regression. Can you give it a go?

Can I restart one node at a time?

Also, does this affect constant_score as well as match_all?

thanks

clint

-shay.banon

On Tue, Oct 26, 2010 at 9:47 AM, Paul ppearcy@gmail.com wrote:
Hey Clint,
I can confirm that this is the only slow down we have run
into. I've
pumped millions of real world queries through since, mostly
consisting
of complex nested fielded boolean queries sorted by date and
there are
no other performance differences. I'm definitely only touching
a
fraction of the search functionality, though.

    Thanks,
    Paul
    
    
    On Oct 25, 10:51 pm, Clinton Gormley <clin...@iannounce.co.uk>
    wrote:
    > On Mon, 2010-10-25 at 16:58 -0700, Paul wrote:
    > > Further update, by removing the unseeded *:* I have
    confirmed that
    > > search times are blazing again.
    >
    > Paul - please keep an eye out for other slowdowns - I am
    seeing some
    > very slow responses on 0.12 for queries which were working
    well before.
    >
    > clint
    >
    > --
    > Web Announcements Limited is a company registered in England
    and Wales,
    > with company number 05608868, with registered address at 10
    Arvon Road,
    > London, N5 1PR.

--
Web Announcements Limited is a company registered in England and Wales,
with company number 05608868, with registered address at 10 Arvon Road,
London, N5 1PR.


(Shay Banon) #8

Yes, you can restart one node at a time (i.e. a rolling upgrade). It does
not affect constant score, only places where one uses match_all
(query/filter) or :.

-shay.banon

On Tue, Oct 26, 2010 at 12:12 PM, Clinton Gormley
clinton@iannounce.co.ukwrote:

Hi kimchy

I added an optimization in match_all cases (including query string
like :), but, that optimization has resulted in excessive processing
in some cases. Easily fixable while still maintaing the optimization.
I have created a branch for 0.12 on github with the fix:
http://github.com/elasticsearch/elasticsearch/tree/0.12. I will
release 0.12.1 pretty soon (likely today) as this is a major
regression. Can you give it a go?

Can I restart one node at a time?

Also, does this affect constant_score as well as match_all?

thanks

clint

-shay.banon

On Tue, Oct 26, 2010 at 9:47 AM, Paul ppearcy@gmail.com wrote:
Hey Clint,
I can confirm that this is the only slow down we have run
into. I've
pumped millions of real world queries through since, mostly
consisting
of complex nested fielded boolean queries sorted by date and
there are
no other performance differences. I'm definitely only touching
a
fraction of the search functionality, though.

    Thanks,
    Paul


    On Oct 25, 10:51 pm, Clinton Gormley <clin...@iannounce.co.uk>
    wrote:
    > On Mon, 2010-10-25 at 16:58 -0700, Paul wrote:
    > > Further update, by removing the unseeded *:* I have
    confirmed that
    > > search times are blazing again.
    >
    > Paul - please keep an eye out for other slowdowns - I am
    seeing some
    > very slow responses on 0.12 for queries which were working
    well before.
    >
    > clint
    >
    > --
    > Web Announcements Limited is a company registered in England
    and Wales,
    > with company number 05608868, with registered address at 10
    Arvon Road,
    > London, N5 1PR.

--
Web Announcements Limited is a company registered in England and Wales,
with company number 05608868, with registered address at 10 Arvon Road,
London, N5 1PR.


(system) #9