Elasticsearch GC allocation failure errors in GC.log

Hi,

I have allocated 32GB for elasticsearch heap memory and keep seeing below error in gc.log.

**[2020-07-07T14:44:44.064+0000][1][gc,start    ] GC(87) Pause Young (Allocation Failure)**
**[2020-07-07T14:44:44.064+0000][1][gc,task     ] GC(87) Using 23 workers of 23 for evacuation**
**[2020-07-07T14:44:44.100+0000][1][gc,age      ] GC(87) Desired survivor size 100302848 bytes, new threshold 6 (max threshold 6)**
**[2020-07-07T14:44:44.100+0000][1][gc,age      ] GC(87) Age table with threshold 6 (max threshold 6)**
**[2020-07-07T14:44:44.101+0000][1][gc,age      ] GC(87) - age   1:   53297824 bytes,   53297824 total**
**[2020-07-07T14:44:44.101+0000][1][gc,age      ] GC(87) - age   2:     923216 bytes,   54221040 total**
**[2020-07-07T14:44:44.101+0000][1][gc,age      ] GC(87) - age   3:   34670888 bytes,   88891928 total**
**[2020-07-07T14:44:44.101+0000][1][gc,age      ] GC(87) - age   4:     832520 bytes,   89724448 total**
**[2020-07-07T14:44:44.101+0000][1][gc,age      ] GC(87) - age   5:     310616 bytes,   90035064 total**
**[2020-07-07T14:44:44.101+0000][1][gc,heap     ] GC(87) ParNew: 1606580K->90434K(1763584K)**
**[2020-07-07T14:44:44.101+0000][1][gc,heap     ] GC(87) CMS: 1114731K->1114752K(31594944K)**
**[2020-07-07T14:44:44.101+0000][1][gc,metaspace] GC(87) Metaspace: 92977K->92977K(98304K)**
**[2020-07-07T14:44:44.101+0000][1][gc          ] GC(87) Pause Young (Allocation Failure) 2657M->1176M(32576M) 36.459ms**
**[2020-07-07T14:44:44.101+0000][1][gc,cpu      ] GC(87) User=0.54s Sys=0.01s Real=0.03s**
[2020-07-07T14:44:44.101+0000][1][safepoint   ] Safepoint "GenCollectForAllocation", Time since last: 137006990733 ns, Reaching safepoint: 588483 ns, At safepoint: 36656203 ns, Total: 37244686 ns
[2020-07-07T14:45:52.109+0000][1][safepoint   ] Safepoint "Cleanup", Time since last: 68007487806 ns, Reaching safepoint: 498214 ns, At safepoint: 23898 ns, Total: 522112 ns
[2020-07-07T14:49:31.043+0000][1][safepoint   ] Safepoint "RevokeBias", Time since last: 218933284318 ns, Reaching safepoint: 416420 ns, At safepoint: 54571 ns, Total: 470991 ns
[2020-07-07T14:52:31.064+0000][1][safepoint   ] Safepoint "Cleanup", Time since last: 180020721722 ns, Reaching safepoint: 414080 ns, At safepoint: 31734 ns, Total: 445814 ns
[2020-07-07T14:52:32.064+0000][1][safepoint   ] Safepoint "Cleanup", Time since last: 1000141207 ns, Reaching safepoint: 353216 ns, At safepoint: 21641 ns, Total: 374857 ns
[2020-07-07T14:52:35.065+0000][1][safepoint   ] Safepoint "Cleanup", Time since last: 3000395791 ns, Reaching safepoint: 333460 ns, At safepoint: 19149 ns, Total: 352609 ns
[2020-07-07T14:53:30.071+0000][1][safepoint   ] Safepoint "Cleanup", Time since last: 55005799174 ns, Reaching safepoint: 535089 ns, At safepoint: 51808 ns, Total: 586897 ns
[2020-07-07T14:58:05.108+0000][1][safepoint   ] Safepoint "Cleanup", Time since last: 275036314861 ns, Reaching safepoint: 555094 ns, At safepoint: 49822 ns, Total: 604916 ns
[2020-07-07T14:58:15.110+0000][1][safepoint   ] Safepoint "Cleanup", Time since last: 10001662406 ns, Reaching safepoint: 357982 ns, At safepoint: 22417 ns, Total: 380399 ns
[2020-07-07T15:04:42.160+0000][1][safepoint   ] Safepoint "Cleanup", Time since last: 387048797418 ns, Reaching safepoint: 603070 ns, At safepoint: 48593 ns, Total: 651663 ns
[2020-07-07T15:04:59.153+0000][1][safepoint   ] Safepoint "RevokeBias", Time since last: 16992759100 ns, Reaching safepoint: 622815 ns, At safepoint: 126143 ns, Total: 748958 ns
[2020-07-07T15:04:59.154+0000][1][safepoint   ] Safepoint "RevokeBias", Time since last: 207924 ns, Reaching safepoint: 563466 ns, At safepoint: 111496 ns, Total: 674962 ns
[2020-07-07T15:04:59.155+0000][1][safepoint   ] Safepoint "RevokeBias", Time since last: 382054 ns, Reaching safepoint: 295875 ns, At safepoint: 77116 ns, Total: 372991 ns
[2020-07-07T15:04:59.155+0000][1][safepoint   ] Safepoint "RevokeBias", Time since last: 145871 ns, Reaching safepoint: 233854 ns, At safepoint: 65626 ns, Total: 299480 ns
[2020-07-07T15:04:59.156+0000][1][safepoint   ] Safepoint "RevokeBias", Time since last: 193046 ns, Reaching safepoint: 251950 ns, At safepoint: 66820 ns, Total: 318770 ns
[2020-07-07T15:04:59.156+0000][1][safepoint   ] Safepoint "RevokeBias", Time since last: 118374 ns, Reaching safepoint: 229501 ns, At safepoint: 65256 ns, Total: 294757 ns
[2020-07-07T15:04:59.157+0000][1][safepoint   ] Safepoint "RevokeBias", Time since last: 112988 ns, Reaching safepoint: 244463 ns, At safepoint: 65431 ns, Total: 309894 ns
[2020-07-07T15:04:59.157+0000][1][safepoint   ] Safepoint "RevokeBias", Time since last: 131924 ns, Reaching safepoint: 234764 ns, At safepoint: 65233 ns, Total: 299997 ns
[2020-07-07T15:04:59.159+0000][1][safepoint   ] Safepoint "RevokeBias", Time since last: 1506749 ns, Reaching safepoint: 365135 ns, At safepoint: 130695 ns, Total: 495830 ns
[2020-07-07T15:04:59.160+0000][1][safepoint   ] Safepoint "RevokeBias", Time since last: 565326 ns, Reaching safepoint: 421292 ns, At safepoint: 168477 ns, Total: 589769 ns
[2020-07-07T15:04:59.161+0000][1][safepoint   ] Safepoint "RevokeBias", Time since last: 483362 ns, Reaching safepoint: 491277 ns, At safepoint: 118526 ns, Total: 609803 ns
[2020-07-07T15:04:59.164+0000][1][safepoint   ] Safepoint "RevokeBias", Time since last: 2460275 ns, Reaching safepoint: 316285 ns, At safepoint: 106505 ns, Total: 422790 ns
[2020-07-07T15:04:59.190+0000][1][safepoint   ] Safepoint "RevokeBias", Time since last: 24999330 ns, Reaching safepoint: 401598 ns, At safepoint: 129140 ns, Total: 530738 ns
[2020-07-07T15:05:00.191+0000][1][safepoint   ] Safepoint "Cleanup", Time since last: 1000218467 ns, Reaching safepoint: 617580 ns, At safepoint: 47313 ns, Total: 664893 ns
[2020-07-07T15:05:02.192+0000][1][safepoint   ] Safepoint "Cleanup", Time since last: 2000305120 ns, Reaching safepoint: 548981 ns, At safepoint: 39106 ns, Total: 588087 ns
[2020-07-07T15:05:15.193+0000][1][safepoint   ] Safepoint "Cleanup", Time since last: 13001336351 ns, Reaching safepoint: 376478 ns, At safepoint: 41324 ns, Total: 417802 ns
[2020-07-07T15:06:51.204+0000][1][safepoint   ] Safepoint "Cleanup", Time since last: 96009650943 ns, Reaching safepoint: 692712 ns, At safepoint: 46415 ns, Total: 739127 ns
[2020-07-07T15:07:07.206+0000][1][safepoint   ] Safepoint "Cleanup", Time since last: 16002211919 ns, Reaching safepoint: 422171 ns, At safepoint: 22085 ns, Total: 444256 ns
[2020-07-07T15:08:05.509+0000][1][safepoint   ] Safepoint "RevokeBias", Time since last: 58302426315 ns, Reaching safepoint: 425881 ns, At safepoint: 79408 ns, Total: 505289 ns
**[2020-07-07T15:15:09.843+0000][1][gc,start    ] GC(88) Pause Young (Allocation Failure)**
**[2020-07-07T15:15:09.844+0000][1][gc,task     ] GC(88) Using 23 workers of 23 for evacuation**
**[2020-07-07T15:15:09.875+0000][1][gc,age      ] GC(88) Desired survivor size 100302848 bytes, new threshold 4 (max threshold 6)**
**[2020-07-07T15:15:09.875+0000][1][gc,age      ] GC(88) Age table with threshold 4 (max threshold 6)**
**[2020-07-07T15:15:09.875+0000][1][gc,age      ] GC(88) - age   1:   34714512 bytes,   34714512 total**
**[2020-07-07T15:15:09.875+0000][1][gc,age      ] GC(88) - age   2:   53233992 bytes,   87948504 total**
**[2020-07-07T15:15:09.875+0000][1][gc,age      ] GC(88) - age   3:     923216 bytes,   88871720 total**
**[2020-07-07T15:15:09.875+0000][1][gc,age      ] GC(88) - age   4:   34669904 bytes,  123541624 total**
**[2020-07-07T15:15:09.875+0000][1][gc,age      ] GC(88) - age   5:     832480 bytes,  124374104 total**
**[2020-07-07T15:15:09.875+0000][1][gc,age      ] GC(88) - age   6:     310616 bytes,  124684720 total**
**[2020-07-07T15:15:09.875+0000][1][gc,heap     ] GC(88) ParNew: 1658114K->125227K(1763584K)**
**[2020-07-07T15:15:09.875+0000][1][gc,heap     ] GC(88) CMS: 1114752K->1114752K(31594944K)**
**[2020-07-07T15:15:09.875+0000][1][gc,metaspace] GC(88) Metaspace: 93210K->93210K(98304K)**
**[2020-07-07T15:15:09.875+0000][1][gc          ] GC(88) Pause Young (Allocation Failure) 2707M->1210M(32576M) 31.693ms**
**[2020-07-07T15:15:09.875+0000][1][gc,cpu      ] GC(88) User=0.55s Sys=0.01s Real=0.03s**

Please suggest how to overcome the above allocation failure error.

Thanks,
Sunny

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.