Increased QPS erratic CPU behavior

Hi !

I'm actually testing ElasticSearch 0.90.x and go very
erratic behavior while increasing query rate. We are running many
benchmarks to push the 2 nodes cluster to the limit, with our standard
query which is consistent mainly of a querystring + terms filter.

Basically, when we run the cluster at a query rate of <26 everything seems
fine, the cpu usage is at ~50%, no iowait at all and pretty decent response
time (~150ms). But thats when we go up to 27 QPS, the CPU usage seems to go
through the roof (100%), still no IO wait but the response time goes up to
1000ms.

While doing some CPU profiling, we noticed that the FixedBitSet methods
(or, and, ) usage seems to have grown a lot (about 10x more). The 2
cpu snapshots are attached to this message.

So, how come that an increase of 1 query per second can completely
overwhelm the cluster like that ? There is nothing else changed, only an
increase of the rate into our benchmark. This seems like a weird behavior
and it's kind of blocking at the moment.

--
Jérôme Gagnon

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.