Got thread waiting in java when search

Elastic Stack Elasticsearch
1

Hi,

I got this issue when build a basic search service by ElasticSearch in
Java as belown:

SearchResponse resp = client
.prepareSearch(ElasticSearchManager.INDEX_NAME)
.setTypes(ElasticSearchManager.INDEX_TYPE)
// way 1
// .setQuery(QueryBuilders.termQuery("pkgName",
// pkgName.toLowerCase()))
// way 2
.setQuery(QueryBuilders.matchAllQuery())
.setPostFilter(
FilterBuilders.termFilter("pkgName",
pkgName.toLowerCase())).execute().actionGet
();

And this method is invoked in a for loop on a list with 20 elements .

When we run this service in Tomcat container under 500 concurrent thread,
so many threads are waiting like this(TPS is about 1500):
名称: catalina-exec-15
状态: org.elasticsearch.common.util.concurrent.BaseFuture$Sync@23772759上的
WAITING
总阻止数: 1,136, 总等待数: 166,545

堆栈跟踪:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(
AbstractQueuedSynchronizer.java:834)
java.util.concurrent.locks.AbstractQueuedSynchronizer.
doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
java.util.concurrent.locks.AbstractQueuedSynchronizer.
acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
org.elasticsearch.common.util.concurrent.BaseFuture$Sync.get(BaseFuture.java
:274)
org.elasticsearch.common.util.concurrent.BaseFuture.get(BaseFuture.java:113)
org.elasticsearch.action.support.AdapterActionFuture.actionGet(
AdapterActionFuture.java:45)
com.xxx.service.elastic.ElasticSearchService.getByPkgName(
ElasticSearchService.java:324)
... ...

(sorry for the Chinese system language )

monitor info from jvisualvm:

Notes:

  1. All ElasticsSearch is default .
  2. Another service without for loop is really good , 10000 concurrent with
    5000+ tps .

So is for loop the real reason ? How should I update this ?
Really thanks for your attention .

nodexy

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b0b4c52d-7e6d-4ad9-8cd9-261630f2c478%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Fixed this .

The key point is NEVER invoke cross-server or cross-process service in a
for loop . This is not a question to elasticsearch but common programming
sutff.
Sorry for disturbation. Thanks.

On Tuesday, December 2, 2014 5:55:21 PM UTC+8, nodexy wrote:

Hi,

I got this issue when build a basic search service by Elasticsearch in
Java as belown:

SearchResponse resp = client
.prepareSearch(ElasticSearchManager.INDEX_NAME)
.setTypes(ElasticSearchManager.INDEX_TYPE)
// way 1
// .setQuery(QueryBuilders.termQuery("pkgName",
// pkgName.toLowerCase()))
// way 2
.setQuery(QueryBuilders.matchAllQuery())
.setPostFilter(
FilterBuilders.termFilter("pkgName",
pkgName.toLowerCase())).execute().
actionGet();

And this method is invoked in a for loop on a list with 20 elements .

When we run this service in Tomcat container under 500 concurrent thread,
so many threads are waiting like this(TPS is about 1500):
名称: catalina-exec-15
状态: org.elasticsearch.common.util.concurrent.BaseFuture$Sync@23772759上的
WAITING
总阻止数: 1,136, 总等待数: 166,545

堆栈跟踪:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
java.util.concurrent.locks.AbstractQueuedSynchronizer.
parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
java.util.concurrent.locks.AbstractQueuedSynchronizer.
doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
java.util.concurrent.locks.AbstractQueuedSynchronizer.
acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
org.elasticsearch.common.util.concurrent.BaseFuture$Sync.get(BaseFuture.
java:274)
org.elasticsearch.common.util.concurrent.BaseFuture.get(BaseFuture.java:
113)
org.elasticsearch.action.support.AdapterActionFuture.actionGet(
AdapterActionFuture.java:45)
com.xxx.service.elastic.ElasticSearchService.getByPkgName(
ElasticSearchService.java:324)
... ...

(sorry for the Chinese system language )

monitor info from jvisualvm:

Notes:

  1. All ElasticsSearch is default .
  2. Another service without for loop is really good , 10000 concurrent with
    5000+ tps .

So is for loop the real reason ? How should I update this ?
Really thanks for your attention .

nodexy

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/bf1ec0db-434a-42a9-a549-934ca7ba047c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

3

I do not have for-loop but I have very much same behavior with threads getting blocked in my tomcat server.

What is the solution here ? I am using 2.0.1 version.