GC (Allocation Failure)

Hi,
Elastisearch was restarted for some reason and unable to narrow down the reason. Please help.

Logstash
[logstash.outputs.elasticsearch] Marking url as dead. Last error: [LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError] Elasticsearch Unreachable: [http://127.0.0.1:9200/][Manticore::SocketException] Software caused connection abort: recv failed {:url=>http://127.0.0.1:9200/, :error_message=>"Elasticsearch Unreachable: [http://127.0.0.1:9200/][Manticore::SocketException] Software caused connection abort: recv failed", :error_class=>"LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError"}

ES : GC log
2019-05-31T09:36:44.955+0400: 597837.381: [GC (Allocation Failure) 2019-05-31T09:36:44.955+0400: 597837.381: [ParNew
Desired survivor size 17432576 bytes, new threshold 6 (max 6)

  • age 1: 3161688 bytes, 3161688 total
  • age 2: 684208 bytes, 3845896 total
  • age 3: 929952 bytes, 4775848 total
  • age 4: 768664 bytes, 5544512 total
    : 286011K->8582K(306688K), 0.0103396 secs] 948944K->671515K(1014528K), 0.0105488 secs] [Times: user=0.00 sys=0.00, real=0.00 secs]
    2019-05-31T09:36:44.955+0400: 597837.391: Total time for which application threads were stopped: 0.0114828 seconds, Stopping threads took: 0.0000692 seconds
    2019-05-31T09:36:45.283+0400: 597837.715: Total time for which application threads were stopped: 0.0007354 seconds, Stopping threads took: 0.0001474 seconds
    2019-05-31T09:36:45.283+0400: 597837.716: Total time for which application threads were stopped: 0.0004456 seconds, Stopping threads took: 0.0000981 seconds
    2019-05-31T09:36:45.283+0400: 597837.717: Total time for which application threads were stopped: 0.0004534 seconds, Stopping threads took: 0.0001133 seconds
    2019-05-31T09:36:45.283+0400: 597837.717: Total time for which application threads were stopped: 0.0004468 seconds, Stopping threads took: 0.0001086 seconds
    2019-05-31T09:36:45.283+0400: 597837.718: Total time for which application threads were stopped: 0.0004270 seconds, Stopping threads took: 0.0000884 seconds
    2019-05-31T09:36:45.283+0400: 597837.719: Total time for which application threads were stopped: 0.0004307 seconds, Stopping threads took: 0.0000887 seconds
    2019-05-31T09:36:45.283+0400: 597837.719: Total time for which application threads were stopped: 0.0004651 seconds, Stopping threads took: 0.0000490 seconds
    2019-05-31T09:36:45.283+0400: 597837.720: Total time for which application threads were stopped: 0.0003739 seconds, Stopping threads took: 0.0000348 seconds
    2019-05-31T09:36:45.283+0400: 597837.721: Total time for which application threads were stopped: 0.0004918 seconds, Stopping threads took: 0.0001406 seconds
    2019-05-31T09:36:45.299+0400: 597837.735: Total time for which application threads were stopped: 0.0010507 seconds, Stopping threads took: 0.0004546 seconds
    2019-05-31T09:36:45.299+0400: 597837.736: Total time for which application threads were stopped: 0.0005554 seconds, Stopping threads took: 0.0001502 seconds
    2019-05-31T09:36:45.299+0400: 597837.738: Total time for which application threads were stopped: 0.0005942 seconds, Stopping threads took: 0.0002302 seconds
    2019-05-31T09:36:45.314+0400: 597837.739: Total time for which application threads were stopped: 0.0006079 seconds, Stopping threads took: 0.0002576 seconds
    2019-05-31T09:36:45.314+0400: 597837.741: Total time for which application threads were stopped: 0.0008896 seconds, Stopping threads took: 0.0005027 seconds
    2019-05-31T09:36:45.314+0400: 597837.743: Total time for which application threads were stopped: 0.0005201 seconds, Stopping threads took: 0.0001396 seconds
    2019-05-31T09:36:45.314+0400: 597837.743: Total time for which application threads were stopped: 0.0005126 seconds, Stopping threads took: 0.0001288 seconds
    2019-05-31T09:36:45.314+0400: 597837.744: Total time for which application threads were stopped: 0.0005098 seconds, Stopping threads took: 0.0001331 seconds
    2019-05-31T09:36:45.314+0400: 597837.750: Total time for which application threads were stopped: 0.0007317 seconds, Stopping threads took: 0.0003392 seconds
    2019-05-31T09:36:45.314+0400: 597837.750: Total time for which application threads were stopped: 0.0004679 seconds, Stopping threads took: 0.0000993 seconds
    2019-05-31T09:36:46.267+0400: 597838.696: Total time for which application threads were stopped: 0.0004925 seconds, Stopping threads took: 0.0000984 seconds
    2019-05-31T09:36:46.267+0400: 597838.697: Total time for which application threads were stopped: 0.0001924 seconds, Stopping threads took: 0.0000478 seconds
    2019-05-31T09:36:46.299+0400: 597838.723: Total time for which application threads were stopped: 0.0003913 seconds, Stopping threads took: 0.0000611 seconds
    2019-05-31T09:36:46.439+0400: 597838.865: Total time for which application threads were stopped: 0.0006442 seconds, Stopping threads took: 0.0000779 seconds
    2019-05-31T09:36:46.455+0400: 597838.888: Total time for which application threads were stopped: 0.0002771 seconds, Stopping threads took: 0.0001055 seconds
    Heap
    par new generation total 306688K, used 86572K [0x00000000c0000000, 0x00000000d4cc0000, 0x00000000d4cc0000)
    eden space 272640K, 28% used [0x00000000c0000000, 0x00000000c4c29828, 0x00000000d0a40000)
    from space 34048K, 25% used [0x00000000d0a40000, 0x00000000d12a1980, 0x00000000d2b80000)
    to space 34048K, 0% used [0x00000000d2b80000, 0x00000000d2b80000, 0x00000000d4cc0000)
    concurrent mark-sweep generation total 707840K, used 662932K [0x00000000d4cc0000, 0x0000000100000000, 0x0000000100000000)
    Metaspace used 66479K, capacity 71915K, committed 72192K, reserved 1112064K
    class space used 7918K, capacity 9469K, committed 9508K, reserved 1048576K

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.