The default thread_pool settings did change from RC1 to RC2. See:
You might want to change the thread_pool settings for search thread pool in
your case.
Did your node immediately get search requests after you started your nodes?
Also do you have search requests that require you to use the field data
cache (like faceting, sorting by field or scripting)?
Python] loaded [lang-groovy], sites [head]
[2013-04-15 11:25:06,641][INFO ][node ] [Princess
Python] {0.90.0.RC2}[364]: initialized
[2013-04-15 11:25:06,641][INFO ][node ] [Princess
Python] {0.90.0.RC2}[364]: starting ...
[2013-04-15 11:25:07,396][INFO ][transport ] [Princess
Python] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/
10.50.1.132:9300]}
[2013-04-15 11:25:11,707][INFO ][cluster.service ] [Princess
Python] new_master [Princess
Python][WZCyUhs6QuKJgyPcm0wuFA][inet[/10.50.1.132:9300]], reason:
zen-disco-join (elected_as_master)
[2013-04-15 11:25:11,719][INFO ][discovery ] [Princess
Python] elasticsearch/WZCyUhs6QuKJgyPcm0wuFA
[2013-04-15 11:25:12,423][INFO ][http ] [Princess
Python] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/
10.50.1.132:9200]}
[2013-04-15 11:25:12,423][INFO ][node ] [Princess
Python] {0.90.0.RC2}[364]: started
[2013-04-15 11:25:12,568][INFO ][gateway ] [Princess
Python] recovered [1] indices into cluster_state
[2013-04-15 11:25:16,853][DEBUG][action.search.type ] [Princess
Python] [ads][2]: Failed to execute
[org.elasticsearch.action.search.SearchRequest@7cd62702] while moving to
second phase
org.elasticsearch.common.util.concurrent.EsRejectedExecutionException:
rejected execution of
[org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction$2]
at
org.elasticsearch.common.util.concurrent.EsAbortPolicy.rejectedExecution(EsAbortPolicy.java:35)
at
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.moveToSecondPhase(TransportSearchQueryThenFetchAction.java:133)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:229)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$3.onResult(TransportSearchTypeAction.java:208)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$3.onResult(TransportSearchTypeAction.java:205)
at
org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:142)
at
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:205)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:192)
at
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:178)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Environment:
windows 7
jdk 1.7.0_17, 64-Bit Server VM (build 23.7-b01, mixed mode)
elasticsearch 0.90.0.RC2
lang-groovy plugin 1.4.0
Thanks,
Mihai
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.
I would recommend using the warming api to warm the field data cache:
I think your case the first bulk of incoming search request was busy
leading field data. I guess this was the reason you ran out of threads so
quickly. You can just register a warming query with your index that just
facet / sorts on all fields you also have in your real search requests.
This query doesn't need to return any sensible results, but it just need to
trigger loading of the field data cache.
Python] loaded [lang-groovy], sites [head]
[2013-04-15 11:25:06,641][INFO ][node ] [Princess
Python] {0.90.0.RC2}[364]: initialized
[2013-04-15 11:25:06,641][INFO ][node ] [Princess
Python] {0.90.0.RC2}[364]: starting ...
[2013-04-15 11:25:07,396][INFO ][transport ] [Princess
Python] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/
10.50.1.132:9300]}
[2013-04-15 11:25:11,707][INFO ][cluster.service ] [Princess
Python] new_master [Princess Python][WZCyUhs6QuKJgyPcm0wuFA][inet[/
10.50.1.132:9300]], reason: zen-disco-join (elected_as_master)
[2013-04-15 11:25:11,719][INFO ][discovery ] [Princess
Python] elasticsearch/**WZCyUhs6QuKJgyPcm0wuFA
[2013-04-15 11:25:12,423][INFO ][http ] [Princess
Python] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/
10.50.1.132:9200]}
[2013-04-15 11:25:12,423][INFO ][node ] [Princess
Python] {0.90.0.RC2}[364]: started
[2013-04-15 11:25:12,568][INFO ][gateway ] [Princess
Python] recovered [1] indices into cluster_state
[2013-04-15 11:25:16,853][DEBUG][action.**search.type ]
[Princess Python] [ads][2]: Failed to execute [org.elasticsearch.action.
**search.SearchRequest@7cd62702] while moving to second phase
org.elasticsearch.common.util.*concurrent.EsRejectedExecutionException:
rejected execution of [org.elasticsearch.action.search.type.
TransportSearchQueryThenFetchAction$AsyncAction$2]
at org.elasticsearch.common.util.concurrent.EsAbortPolicy.
rejectedExecution(EsAbortPolicy.java:35)
at java.util.concurrent.ThreadPoolExecutor.reject(
ThreadPoolExecutor.java:821)
at java.util.concurrent.ThreadPoolExecutor.execute(
ThreadPoolExecutor.java:1372)
at org.elasticsearch.action.search.type.
TransportSearchQueryThenFetchAction$AsyncAction.moveToSecondPhase(
*TransportSearchQueryThenFetchAction.java:133)
at org.elasticsearch.action.**search.type.**TransportSearchTypeAction$
**BaseAsyncAction.**onFirstPhaseResult(TransportSearchTypeAction.
java:229)
at org.elasticsearch.action.**search.type.**TransportSearchTypeAction$
**BaseAsyncAction$3.onResult(**TransportSearchTypeAction.**java:208)
at org.elasticsearch.action.**search.type.**TransportSearchTypeAction$
**BaseAsyncAction$3.onResult(**TransportSearchTypeAction.**java:205)
at org.elasticsearch.search.**action.SearchServiceTransportAction.
sendExecuteQuery(SearchServiceTransportAction.java:142)
at org.elasticsearch.action.search.type.
TransportSearchQueryThenFetchAction$AsyncAction.
sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
at org.elasticsearch.action.**search.type.**TransportSearchTypeAction$
**BaseAsyncAction.**performFirstPhase(TransportSearchTypeAction.
java:205)
at org.elasticsearch.action.**search.type.**TransportSearchTypeAction$
**BaseAsyncAction.**performFirstPhase(TransportSearchTypeAction.
java:192)
at org.elasticsearch.action.**search.type.**TransportSearchTypeAction$
**BaseAsyncAction$2.run(**TransportSearchTypeAction.**java:178)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.**java:722)
Environment:
windows 7
jdk 1.7.0_17, 64-Bit Server VM (build 23.7-b01, mixed mode)
elasticsearch 0.90.0.RC2
lang-groovy plugin 1.4.0
Thanks,
Mihai
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.com.
My type has a lot of fields and the user can search against all or a subset
of them (therefore, I don't know the exact query till the user action). Do
you still recommend the warmer in this case?
Thanks!
On Monday, April 15, 2013 3:59:55 PM UTC+3, Martijn v Groningen wrote:
I think your case the first bulk of incoming search request was busy
leading field data. I guess this was the reason you ran out of threads so
quickly. You can just register a warming query with your index that just
facet / sorts on all fields you also have in your real search requests.
This query doesn't need to return any sensible results, but it just need to
trigger loading of the field data cache.
On 15 April 2013 14:44, Mihai Cazacu <cazacu...@gmail.com <javascript:>>wrote:
Thanks, Martijn!
On Monday, April 15, 2013 3:39:34 PM UTC+3, Martijn v Groningen wrote:
Python] loaded [lang-groovy], sites [head]
[2013-04-15 11:25:06,641][INFO ][node ] [Princess
Python] {0.90.0.RC2}[364]: initialized
[2013-04-15 11:25:06,641][INFO ][node ] [Princess
Python] {0.90.0.RC2}[364]: starting ...
[2013-04-15 11:25:07,396][INFO ][transport ] [Princess
Python] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/
10.50.1.132:9300]}
[2013-04-15 11:25:11,707][INFO ][cluster.service ] [Princess
Python] new_master [Princess Python][*WZCyUhs6QuKJgyPcm0wuFA][inet[/
*10.50.1.132:9300]], reason: zen-disco-join (elected_as_master)
[2013-04-15 11:25:11,719][INFO ][discovery ] [Princess
Python] elasticsearch/**WZCyUhs6QuKJgyPcm0wuFA
[2013-04-15 11:25:12,423][INFO ][http ] [Princess
Python] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/
10.50.1.132:9200]}
[2013-04-15 11:25:12,423][INFO ][node ] [Princess
Python] {0.90.0.RC2}[364]: started
[2013-04-15 11:25:12,568][INFO ][gateway ] [Princess
Python] recovered [1] indices into cluster_state
[2013-04-15 11:25:16,853][DEBUG][action.**search.type ]
[Princess Python] [ads][2]: Failed to execute [org.elasticsearch.action.
**search.SearchRequest@7cd62702] while moving to second phase
org.elasticsearch.common.util.**concurrent.EsRejectedExecutionException:
rejected execution of [org.elasticsearch.action.search.type.
TransportSearchQueryThenFetchAction$AsyncAction$2]
at org.elasticsearch.common.util.concurrent.EsAbortPolicy.
rejectedExecution(EsAbortPolicy.java:35)
at java.util.concurrent.ThreadPoolExecutor.reject(
ThreadPoolExecutor.java:821)
at java.util.concurrent.ThreadPoolExecutor.execute(
ThreadPoolExecutor.java:1372)
at org.elasticsearch.action.search.type.
TransportSearchQueryThenFetchAction$AsyncAction.**moveToSecondPhase( TransportSearchQueryThenFetchAction.java:133)
at org.elasticsearch.action.search.type.
TransportSearchTypeAction$**BaseAsyncAction.onFirstPhaseResult(
TransportSearchTypeAction.**java:229)
at org.elasticsearch.action.search.type.
TransportSearchTypeAction$BaseAsyncAction$3.onResult(
TransportSearchTypeAction.**java:208)
at org.elasticsearch.action.search.type.
TransportSearchTypeAction$BaseAsyncAction$3.onResult(
TransportSearchTypeAction.**java:205)
at org.elasticsearch.search.**action.SearchServiceTransportAction.
sendExecuteQuery(SearchServiceTransportAction.java:142)
at org.elasticsearch.action.search.type.
TransportSearchQueryThenFetchAction$AsyncAction.
sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
at org.elasticsearch.action.search.type.
TransportSearchTypeAction$**BaseAsyncAction.performFirstPhase(
TransportSearchTypeAction.**java:205)
at org.elasticsearch.action.search.type.
TransportSearchTypeAction$**BaseAsyncAction.performFirstPhase(
TransportSearchTypeAction.**java:192)
at org.elasticsearch.action.search.type.
TransportSearchTypeAction$BaseAsyncAction$2.run(
TransportSearchTypeAction.**java:178)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.**java:722)
Environment:
windows 7
jdk 1.7.0_17, 64-Bit Server VM (build 23.7-b01, mixed mode)
elasticsearch 0.90.0.RC2
lang-groovy plugin 1.4.0
Thanks,
Mihai
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.com.
My type has a lot of fields and the user can search against all or a
subset of them (therefore, I don't know the exact query till the user
action). Do you still recommend the warmer in this case?
Thanks!
On Monday, April 15, 2013 3:59:55 PM UTC+3, Martijn v Groningen wrote:
I think your case the first bulk of incoming search request was busy
leading field data. I guess this was the reason you ran out of threads so
quickly. You can just register a warming query with your index that just
facet / sorts on all fields you also have in your real search requests.
This query doesn't need to return any sensible results, but it just need to
trigger loading of the field data cache.
Python] loaded [lang-groovy], sites [head]
[2013-04-15 11:25:06,641][INFO ][node ] [Princess
Python] {0.90.0.RC2}[364]: initialized
[2013-04-15 11:25:06,641][INFO ][node ] [Princess
Python] {0.90.0.RC2}[364]: starting ...
[2013-04-15 11:25:07,396][INFO ][transport ] [Princess
Python] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/
10.50.1.132:9300]}
[2013-04-15 11:25:11,707][INFO ][cluster.service ] [Princess
Python] new_master [Princess Python][WZCyUhs6QuKJgyPcm0wuFA
][inet[/10.50.1.132:9300]], reason: zen-disco-join
(elected_as_master)
[2013-04-15 11:25:11,719][INFO ][discovery ] [Princess
Python] elasticsearch/WZCyUhs6QuKJgyPcm0wuFA
[2013-04-15 11:25:12,423][INFO ][http ] [Princess
Python] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/
10.50.1.132:9200]}
[2013-04-15 11:25:12,423][INFO ][node ] [Princess
Python] {0.90.0.RC2}[364]: started
[2013-04-15 11:25:12,568][INFO ][gateway ] [Princess
Python] recovered [1] indices into cluster_state
[2013-04-15 11:25:16,853][DEBUG][action.search.type ]
[Princess Python] [ads][2]: Failed to execute [org.elasticsearch.action. search.SearchRequest@7cd62702] while moving to second phase
org.elasticsearch.common.util.**concurrent.EsRejectedExecutionException:
rejected execution of [org.elasticsearch.action.search.type.
TransportSearchQueryThenFetchAction$AsyncAction$2]
at org.elasticsearch.common.util.**concurrent.EsAbortPolicy.
rejectedExecution(EsAbortPolicy.java:35)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExe cutor.java:821)
at java.util.concurrent.ThreadPoolExecutor.execute(
ThreadPoolExecutor.java:1372)
at org.elasticsearch.action.search.type.TransportSearchQueryThe nFetchAction$AsyncAction.moveToSecondPhase(
TransportSearchQueryThenFetchAction.java:133)
at org.elasticsearch.action.search.type.**
TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult( TransportSearchTypeAction.java:229)
at org.elasticsearch.action.search.type.
TransportSearchTypeAction$BaseAsyncAction$3.onResult(**
TransportSearchTypeAction.java:208)
at org.elasticsearch.action.search.type.
TransportSearchTypeAction$BaseAsyncAction$3.onResult(**
TransportSearchTypeAction.java:205)
at org.elasticsearch.search.action.
SearchServiceTransportAction**.sendExecuteQuery(SearchServic
eTransportAction.java:142)
at org.elasticsearch.action.search.type.
TransportSearchQueryThenFetchAction$AsyncAction.sendE
xecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
at org.elasticsearch.action.search.type.
TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(
*TransportSearchTypeAction.java:205)
at org.elasticsearch.action.search.type.
TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(
*TransportSearchTypeAction.java:192)
at org.elasticsearch.action.search.type.
TransportSearchTypeActi**on$BaseAsyncAction$2.run(Trans
portSearchTypeAction.java:178)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(**ThreadPoo
**lExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Environment:
windows 7
jdk 1.7.0_17, 64-Bit Server VM (build 23.7-b01, mixed mode)
elasticsearch 0.90.0.RC2
lang-groovy plugin 1.4.0
Thanks,
Mihai
--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.**com.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.