Immediate index warmup after documents are reindexed


(Wing) #1

My index size is around 61.7gb with around 7million documents, 10
shards and 1 replica.

Queries with filters and facets works nicely, well belows 500ms.

I met random slowness (around 1 to 3 seconds) on queries with filters
and facets after documents are reindexed, no matter the queries yield
small results (around 50 to 200 results) or large results. If the
queries run 1 to 2 more times, it becomes fast.

I read about "Percolate API" and it can be used to register queries to
warmup index after reindexing documents? But using the percolate to
find and execute the matched queries still do not guarantee the
indexes are well warmup-ed? Because our indexes are being queried
continuously by others at the same time of reindexing. And we are
using bulk index and it seems difficult to use Percolate here.

So how we can effectively warmup the indexes so that queries will not
get this 1 to 3 seconds slowness?

Wing


(Clinton Gormley) #2

So how we can effectively warmup the indexes so that queries will not
get this 1 to 3 seconds slowness?

Due out in 0.20: warmup API

clint


(Wing) #3

Then, do we have schedule for the launch of 0.20?

Thanks,
Wing

On Fri, Jun 22, 2012 at 3:33 PM, Clinton Gormley clint@traveljury.com wrote:

So how we can effectively warmup the indexes so that queries will not
get this 1 to 3 seconds slowness?

Due out in 0.20: warmup API
https://github.com/elasticsearch/elasticsearch/issues/1913

clint


(Shay Banon) #4

There isn't a fixed data for 0.20 release yet. Though several companies use
it in production.

On Fri, Jun 22, 2012 at 10:58 AM, Yiu Wing TSANG ywtsang@gmail.com wrote:

Then, do we have schedule for the launch of 0.20?

Thanks,
Wing

On Fri, Jun 22, 2012 at 3:33 PM, Clinton Gormley clint@traveljury.com
wrote:

So how we can effectively warmup the indexes so that queries will not
get this 1 to 3 seconds slowness?

Due out in 0.20: warmup API
https://github.com/elasticsearch/elasticsearch/issues/1913

clint


(system) #5