Java Client API to check the size of bulk queue

I am getting EsRejectedExecutionException after sending the logs to the bulkprocessor.

I know the queue size defaults to 200. Is there any Java API with which I can check the current queue size and handle the addition of requests accordingly?

Not sure but I definitely prefer using the BulkProcessor class instead of doing manual stuff... Not sure this helps though.

I am using bulkprocessor class. The problem is that we need to control the rate with which we are sending requests to bulkprocessor because queue is getting filled. So I want to know if there is a way get the current queue size like how we get using the nodestats call over http.

I see. I wish we can enhance at some point the bulk processor to slow down automatically when the cluster gets overloaded. In the same way beats and logstash are doing I think.

Anyway, I don't there is a method to access nodes stats API (I did not find it).
So here is may be a way to get that information. Note that you will have to parse the JSON response with whatever framework (Jackson?) to access the data you want to look for.

Response response = client.getLowLevelClient().performRequest(new Request("GET", "/_nodes/stats/thread_pool"));
String s = EntityUtils.toString(response.getEntity());
System.out.println("thread_pool = " + s);

I have one more doubt regarding this.

By default bulkprocessor
sets backoffPolicy to an exponential backoff with 8 retries and a start delay of 50ms. The total wait time is roughly 5.1 seconds.

So in node _nodes/stats/thread_pool, what happens to the rejected requests in rejected section? Are the retried again and does this number changes if the retried request succeeds?

I don't think this number will change.

NodesStatsResponse nodesStatsResponse = client.admin().cluster().nodesStats(new NodesStatsRequest().all()).actionGet();
	for (NodeStats node : nodesStatsResponse.getNodes()) {
        if (node.getHostname().equalsIgnoreCase("NODENAME")) {
            ThreadPoolStats thread = node.getThreadPool();
            Iterator<ThreadPoolStats.Stats> itr = thread.iterator();
            while (itr.hasNext()) {
                ThreadPoolStats.Stats row =;
                if (row.getName().equalsIgnoreCase("bulk")) {
                    int currentQueueSize = row.getQueue();


I used NodesStatsResponse api to get the bulk queue size.

Please respond if this can be used to get the bulk queue size for a particular node

Note that you are using a TransportClient which is removed in 8.0.

The Nodes Stats API allows you to set a specific node with:

GET /_nodes/nodeId/stats/thread_pool

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.