# GC warnings and Time out exceptions on bulk update

**URL:** <https://discuss.elastic.co/t/gc-warnings-and-time-out-exceptions-on-bulk-update/29212>\
**Category:** Elasticsearch\
**Created:** [September 14, 2015, 4:25am UTC](https://discuss.elastic.co/t/gc-warnings-and-time-out-exceptions-on-bulk-update/29212 "2015-09-14T04:25:35Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![spiral\_circ](https://avatars.discourse-cdn.com/v4/letter/s/e68b1a/32.png) [@spiral\_circ](https://discuss.elastic.co/u/spiral_circ)\
**Post date:** [September 14, 2015, 4:25am UTC](https://discuss.elastic.co/t/gc-warnings-and-time-out-exceptions-on-bulk-update/29212/1 "2015-09-14T04:25:35Z")

</div>

Hi,

I am trying to bulk update about 2.3 million documents in batches of 100K.  
After about 400K - 500K i start seeing the below exceptions on the client side. And i see lot of gc logging on the elastic search console (pasted below).

I am doing this in 2 steps. I get all the ids of the documents using the scroll api (which works fine), and after that do a bulk update in batches of 100K.

If anyone can point if I am doing anything wrong or if there is a better way to do this that would be great.

Client Bulk Update Code :

```
            while (true) {
                count++;
                String id = idQueue.take(); //BlockingQueue<String>, ids obtained using search/scroll api

                updateRequest = new UpdateRequest(index, type, id).script
                        ("ctx._source.TOTAL_COMPENSATION = Double.parseDouble(ctx._source.SALARY) + Double.parseDouble(ctx._source.BONUS)");
                bulkRequest.add(updateRequest);
                if (count % 100000 == 0) {
                    BulkResponse bulkItemResponses = bulkRequest.execute().actionGet();
                    System.out.println("UPDATED " + count + " Took " + bulkItemResponses.getTook());
                    bulkRequest = transportClient.prepareBulk();
                }
            }

```

## Client Log:

UPDATED + 500000 Took 2.3m  
Sep 13, 2015 11:48:36 PM org.elasticsearch.client.transport.TransportClientNodesService$SimpleNodeSampler doSample  
INFO: [Jocasta] failed to get node info for [#transport#-1][localhost][inet[localhost/127.0.0.1:9300]], disconnecting...  
org.elasticsearch.transport.ReceiveTimeoutTransportException: [][inet[localhost/127.0.0.1:9300]][cluster:monitor/nodes/info] request\_id [142] timed out after [5001ms]  
at org.elasticsearch.transport.TransportService$TimeoutHandler.run(TransportService.java:529)  
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:745)

Sep 13, 2015 11:48:38 PM org.elasticsearch.transport.TransportService$Adapter checkForTimeout  
WARNING: [Jocasta] Received response for a request that has timed out, sent [7929ms] ago, timed out [2928ms] ago, action [cluster:monitor/nodes/info], node [[#transport#-1][localhost][inet[localhost/127.0.0.1:9300]]], id [142]  
UPDATED + 600000 Took 2.5m  
Sep 13, 2015 11:49:52 PM org.elasticsearch.client.transport.TransportClientNodesService$SimpleNodeSampler doSample

## ElasticSearch Console:

[2015-09-13 23:54:49,830][WARN][monitor.jvm] [Mad Jack] [gc][young][968][244] duration [2.2s], collections [1]/[3s], total [2.2s]/[5.1m], memory [9.1gb]-\>[8.8gb]/[11.9gb], all\_pools {[young] [468.3mb]-\>[10.7mb]/[532.5mb]}{[survivor] [66.5mb]-\>[66.5mb]/[66.5mb]}{[old] [8.6gb]-\>[8.7gb]/[11.3gb]}  
^C[2015-09-13 23:55:17,172][WARN][monitor.jvm] [Mad Jack] [gc][young][986][245] duration [8.7s], collections [1]/[8.9s], total [8.7s]/[5.3m], memory [9.2gb]-\>[8.9gb]/[11.9gb], all\_pools {[young] [489mb]-\>[944.6kb]/[532.5mb]}{[survivor] [66.5mb]-\>[66.5mb]/[66.5mb]}{[old] [8.7gb]-\>[8.8gb]/[11.3gb]}

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [September 14, 2015, 6:07am UTC](https://discuss.elastic.co/t/gc-warnings-and-time-out-exceptions-on-bulk-update/29212/2 "2015-09-14T06:07:03Z")

</div>

It looks like you're simply pushing data quicker than what your current cluster can handle. Have you tried reducing the batch size? I don't think 100k batches gives better performance than, say, 10k.

---

<div class="post-metadata">

**Author:** ![dubul](https://avatars.discourse-cdn.com/v4/letter/d/9f8e36/32.png) [@dubul](https://discuss.elastic.co/u/dubul)\
**Post date:** [January 5, 2017, 9:08am UTC](https://discuss.elastic.co/t/gc-warnings-and-time-out-exceptions-on-bulk-update/29212/3 "2017-01-05T09:08:06Z")

</div>

i'm getting the same error, how can I check how much data my cluster can handle? Is it something I can modify the elastic search config file or filebeat.yaml?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:04pm UTC](https://discuss.elastic.co/t/gc-warnings-and-time-out-exceptions-on-bulk-update/29212/4 "2017-07-05T22:04:28Z")

</div>