High CPU Hot thread suggest a highlight problem

Elastic Stack Elasticsearch
1

Hello,

I encounter high CPU load without high query or indexation rate.

The outpout of hot_thread API suggest an highlight problem.
A cpoie aof the output is here :

Any suggestions ?
Is it possible to stop (kill) these thread without restarting nodes ?

Thanks.

Benoît

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hi,
can you post the query that is causing this? I guess it's a phrase query,
isn't it? We are aware of issues with certain queries in combination with
fast vector highlighter, and that's what your hot_thread output shows too.
We just reverted some recent changes we made to support complex queries
(Highlighting doesn't work with term vectors enabled and some complex queries · Issue #3357 · elastic/elasticsearch · GitHub), which were
causing issues with fast vector highlighter, even StackOverflowErrors in
some cases. Your issue seems to be related.

I'm not aware of a way to kill those threads without restarting the
elasticsearch process. You can wait and see if their eecution finishes at
some point. What I would suggest is not to use the fast vector highlighter
in your case, at least until we released the next version. You can force
the use of the plain highlighter even though you stored term_vectors using
the "type":"plain" option in your highlight request.

Hope this helps
Luca

On Friday, September 6, 2013 2:36:39 PM UTC+2, Benoît wrote:

Hello,

I encounter high CPU load without high query or indexation rate.

The outpout of hot_thread API suggest an highlight problem.
A cpoie aof the output is here :
/_nodes/hot_threads output · GitHub

Any suggestions ?
Is it possible to stop (kill) these thread without restarting nodes ?

Thanks.

Benoît

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

3

Hi,

On Friday, September 6, 2013 4:19:29 PM UTC+2, Luca Cavanna wrote:

can you post the query that is causing this?

I guess it's a phrase query, isn't it?

Our queries are mostly big, nested and with phrase, usually with distance
(slop)

We are aware of issues with certain queries in combination with fast vector

highlighter, and that's what your hot_thread output shows too. We just
reverted some recent changes we made to support complex queries (
Highlighting doesn't work with term vectors enabled and some complex queries · Issue #3357 · elastic/elasticsearch · GitHub), which were
causing issues with fast vector highlighter, even StackOverflowErrors in
some cases. Your issue seems to be related.

It happened three or two times at a week interval, not sure to be able to
reproduce the problem. I got StackOverflowErrors once

I'm not aware of a way to kill those threads without restarting the
elasticsearch process. You can wait and see if their eecution finishes at
some point.

They run for 5 hours. :-/

What I would suggest is not to use the fast vector highlighter in your
case, at least until we released the next version. You can force the use of
the plain highlighter even though you stored term_vectors using the
"type":"plain" option in your highlight request.

So there is a workaround in next version 0.90.4 which should be available
next week ?

Hope this helps

Yes, thank you very mutch.

Benoît

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

4

Well, we dropped the support for some complex queries, which won't be
highlighted as far as I understand. Trying to highlight them was causing
the nodes to become unstable, which is something we don't want to happen.

We've been thinking about the best way to proceed to support those queries
with the fast vector highlighter without causing the problems you;re
experiencing. The problem is mainly in the underlying lucene and the logic
that the fast vector highlighter uses when it comes to highlight queries
that involve positions of the terms (phrase queries, span queries)

In the meantime I would suggest you to use the plain highlighter.

Cheers
Luca

On Fri, Sep 6, 2013 at 4:42 PM, Benoît benoit.intrw@gmail.com wrote:

Hi,

On Friday, September 6, 2013 4:19:29 PM UTC+2, Luca Cavanna wrote:

can you post the query that is causing this?

I guess it's a phrase query, isn't it?

Our queries are mostly big, nested and with phrase, usually with distance
(slop)

We are aware of issues with certain queries in combination with fast

vector highlighter, and that's what your hot_thread output shows too. We
just reverted some recent changes we made to support complex queries (
https://github.com/**elasticsearch/elasticsearch/**issues/3357https://github.com/elasticsearch/elasticsearch/issues/3357),
which were causing issues with fast vector highlighter, even
StackOverflowErrors in some cases. Your issue seems to be related.

It happened three or two times at a week interval, not sure to be able to
reproduce the problem. I got StackOverflowErrors once

I'm not aware of a way to kill those threads without restarting the
elasticsearch process. You can wait and see if their eecution finishes at
some point.

They run for 5 hours. :-/

What I would suggest is not to use the fast vector highlighter in your
case, at least until we released the next version. You can force the use of
the plain highlighter even though you stored term_vectors using the
"type":"plain" option in your highlight request.

So there is a workaround in next version 0.90.4 which should be available
next week ?

Hope this helps

Yes, thank you very mutch.

Benoît

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/zjb-yP3odqk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.