# Java Client API to check the size of bulk queue

**URL:** https://discuss.elastic.co/t/java-client-api-to-check-the-size-of-bulk-queue/188520
**Category:** Elasticsearch
**Created:** [July 2, 2019, 1:25pm UTC](https://discuss.elastic.co/t/java-client-api-to-check-the-size-of-bulk-queue/188520 "2019-07-02T13:25:03Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![Prashant\_Rana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prashant_rana/32/42952_2.png) [@Prashant\_Rana](https://discuss.elastic.co/u/Prashant_Rana)
#### Post date: [July 2, 2019, 1:25pm UTC](https://discuss.elastic.co/t/java-client-api-to-check-the-size-of-bulk-queue/188520/1 "2019-07-02T13:25:03Z")

</div>

Hi,  
I am getting EsRejectedExecutionException after sending the logs to the bulkprocessor.

I know the queue size defaults to 200. Is there any Java API with which I can check the current queue size and handle the addition of requests accordingly?

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [July 2, 2019, 3:45pm UTC](https://discuss.elastic.co/t/java-client-api-to-check-the-size-of-bulk-queue/188520/2 "2019-07-02T15:45:39Z")

</div>

Not sure but I definitely prefer using the [BulkProcessor class](https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-document-bulk.html#java-rest-high-document-bulk-processor) instead of doing manual stuff... Not sure this helps though.

---

<div class="post-metadata">

### Author: ![Prashant\_Rana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prashant_rana/32/42952_2.png) [@Prashant\_Rana](https://discuss.elastic.co/u/Prashant_Rana)
#### Post date: [July 2, 2019, 5:13pm UTC](https://discuss.elastic.co/t/java-client-api-to-check-the-size-of-bulk-queue/188520/3 "2019-07-02T17:13:28Z")

</div>

I am using bulkprocessor class. The problem is that we need to control the rate with which we are sending requests to bulkprocessor because queue is getting filled. So I want to know if there is a way get the current queue size like how we get using the nodestats call over http.

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [July 2, 2019, 7:38pm UTC](https://discuss.elastic.co/t/java-client-api-to-check-the-size-of-bulk-queue/188520/4 "2019-07-02T19:38:16Z")

</div>

I see. I wish we can enhance at some point the bulk processor to slow down automatically when the cluster gets overloaded. In the same way beats and logstash are doing I think.

Anyway, I don't there is a method to access nodes stats API (I did not find it).  
So here is may be a way to get that information. Note that you will have to parse the JSON response with whatever framework (Jackson?) to access the data you want to look for.

```auto
Response response = client.getLowLevelClient().performRequest(new Request("GET", "/_nodes/stats/thread_pool"));
String s = EntityUtils.toString(response.getEntity());
System.out.println("thread_pool = " + s);

```

---

<div class="post-metadata">

### Author: ![Prashant\_Rana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prashant_rana/32/42952_2.png) [@Prashant\_Rana](https://discuss.elastic.co/u/Prashant_Rana)
#### Post date: [July 3, 2019, 9:54am UTC](https://discuss.elastic.co/t/java-client-api-to-check-the-size-of-bulk-queue/188520/5 "2019-07-03T09:54:56Z")

</div>

I have one more doubt regarding this.

By default bulkprocessor  
`sets backoffPolicy to an exponential backoff with 8 retries and a start delay of 50ms. The total wait time is roughly 5.1 seconds.`

So in node `_nodes/stats/thread_pool`, what happens to the rejected requests in `rejected` section? Are the retried again and does this number changes if the retried request succeeds?

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [July 3, 2019, 11:31am UTC](https://discuss.elastic.co/t/java-client-api-to-check-the-size-of-bulk-queue/188520/6 "2019-07-03T11:31:54Z")

</div>

> [@Prashant\_Rana](#):
>
> does this number changes if the retried request succeeds?

I don't think this number will change.

---

<div class="post-metadata">

### Author: ![Prashant\_Rana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prashant_rana/32/42952_2.png) [@Prashant\_Rana](https://discuss.elastic.co/u/Prashant_Rana)
#### Post date: [July 3, 2019, 12:18pm UTC](https://discuss.elastic.co/t/java-client-api-to-check-the-size-of-bulk-queue/188520/7 "2019-07-03T12:18:47Z")

</div>

```
NodesStatsResponse nodesStatsResponse = client.admin().cluster().nodesStats(new NodesStatsRequest().all()).actionGet();
	for (NodeStats node : nodesStatsResponse.getNodes()) {
        if (node.getHostname().equalsIgnoreCase("NODENAME")) {
            ThreadPoolStats thread = node.getThreadPool();
            Iterator<ThreadPoolStats.Stats> itr = thread.iterator();
            while (itr.hasNext()) {
                ThreadPoolStats.Stats row = itr.next();
                if (row.getName().equalsIgnoreCase("bulk")) {
                    int currentQueueSize = row.getQueue();
                }
            }
        }

    }

```

I used NodesStatsResponse api to get the bulk queue size.

Please respond if this can be used to get the bulk queue size for a particular node

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [July 3, 2019, 1:46pm UTC](https://discuss.elastic.co/t/java-client-api-to-check-the-size-of-bulk-queue/188520/8 "2019-07-03T13:46:25Z")

</div>

Note that you are using a `TransportClient` which is removed in 8.0.

The [Nodes Stats API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-nodes-stats.html) allows you to set a specific node with:

```auto
GET /_nodes/nodeId/stats/thread_pool

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 31, 2019, 1:46pm UTC](https://discuss.elastic.co/t/java-client-api-to-check-the-size-of-bulk-queue/188520/9 "2019-07-31T13:46:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
