Having a few issues with search performance. I was hoping there would be
some way to get a detailed output on what phases of the search took all my
time. I'm planning on passing the slowlog along to Kibana to set some idea
of what is going on on a shard level, but thought I'd put this out there
incase there was any way to get a breakdown along with the query response.
i.e. I had to wait on shard X due to Y. Plus for highlighting any flaws
in what I'm doing.
Setup: 6 data nodes, 256GB, 8xSSD, 60GB heap, G1GC
Some of our queries are quite large. Large near queries, up to 100
terms or'd together, neared with 5 terms, including wildcards. That type
of thing. They take a long time, sometimes upwards of 3 mins. We want to
be able to support this. 1 second responses maybe not, but I'd say 30
seconds at a max..
These slow queries tend to prevent other queries from completing in
sensible times, even for simple queries.
There's a large variance in query times. Sometimes 1 sec, sometimes
much higher.
I've tried to correlate to GCs and merges without success. Ideally I'd
like to know what were queries waiting on, from which I can make
adjustments. Obviously I could add more nodes to the cluster, but I'd like
to know why I'm doing that rather than just taking a punt. Advise/abuse
welcome.
They typically consist of an "entity" neared with a large number of
"negative" (user specified) terms. So something like [bad person near50
(long or list near5 negative or terms ...)] which we translate into a big
horrible near_span query. When we can detect a language then we can apply
one analyzer and query the appropriate fields. Otherwise, we query across
a number of fields with different language analyzed fields. It's not nice.
It's pretty typical in certain industries.
On Thursday, 10 April 2014 00:03:48 UTC+1, hazzadous wrote:
No you don't Binh
On Wednesday, 9 April 2014 17:57:17 UTC+1, Binh Ly wrote:
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
