ElasticSearch high CPU load and Excessive garbage collection

Hi, I am using 3 node cluster in elastic search having configuration 4core 16GB. Heap size allocated for elastic search is 8GB. I have only 3 indices in my cluster having a total of 17lac records (in all 3 indices combined).

I don't do updates on them. Only search queries are performed on them. But I face high CPU usage often in any of the nodes. The version of elastic search, I am using, is 2.4.

Following are my GC logs:-

CommandLine flags: -XX:CMSInitiatingOccupancyFraction=75 -XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError -XX:InitialHeapSize=8589934592 -XX:MaxHeapSize=8589934592 -XX:MaxNewSize=348966912 -XX:MaxTenuringThreshold=6 -XX:NewSize=348966912 -XX:OldPLABSize=16 -XX:OldSize=697933824 -XX:+PrintClassHistogram -XX:+PrintGC -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC

273655K->1054K(306688K), 0.0174304 secs] 1594728K->1322130K(8354560K), 0.0175535 secs] [Times: user=0.07 sys=0.00, real=0.02 secs]
2019-01-08T23:47:41.027+0530: 4855959.377: Total time for which application threads were stopped: 0.0183525 seconds, Stopping threads took: 0.0000959 seconds
2019-01-08T23:49:37.595+0530: 4856075.945: [GC (Allocation Failure) 2019-01-08T23:49:37.595+0530: 4856075.945: [ParNew
Desired survivor size 17432576 bytes, new threshold 6 (max 6)

  • age 1: 724392 bytes, 724392 total
  • age 2: 3920 bytes, 728312 total
  • age 3: 36176 bytes, 764488 total
  • age 4: 4344 bytes, 768832 total
  • age 5: 2400 bytes, 771232 total
  • age 6: 2720 bytes, 773952 total
    : 273694K->1055K(306688K), 0.0185617 secs] 1594770K->1322133K(8354560K), 0.0187332 secs] [Times: user=0.07 sys=0.00, real=0.02 secs]
    2019-01-08T23:49:37.614+0530: 4856075.964: Total time for which application threads were stopped: 0.0196404 seconds, Stopping threads took: 0.0001491 seconds
    2019-01-08T23:51:23.625+0530: 4856181.975: [GC (Allocation Failure) 2019-01-08T23:51:23.625+0530: 4856181.975: [ParNew
    Desired survivor size 17432576 bytes, new threshold 6 (max 6)
  • age 1: 698992 bytes, 698992 total
  • age 2: 39024 bytes, 738016 total
  • age 3: 3864 bytes, 741880 total
  • age 4: 36176 bytes, 778056 total
  • age 5: 4320 bytes, 782376 total
  • age 6: 2400 bytes, 784776 total
    : 273695K->1193K(306688K), 0.0229432 secs] 1594773K->1322273K(8354560K), 0.0230595 secs] [Times: user=0.07 sys=0.00, real=0.02 secs]
    2019-01-08T23:51:23.648+0530: 4856181.999: Total time for which application threads were stopped: 0.0239003 seconds, Stopping threads took: 0.0001023 seconds
    2019-01-08T23:53:21.817+0530: 4856300.167: [GC (Allocation Failure) 2019-01-08T23:53:21.817+0530: 4856300.167: [ParNew
    Desired survivor size 17432576 bytes, new threshold 6 (max 6)
  • age 1: 628344 bytes, 628344 total
  • age 2: 3888 bytes, 632232 total
  • age 3: 38712 bytes, 670944 total
  • age 4: 3864 bytes, 674808 total
  • age 5: 36176 bytes, 710984 total
  • age 6: 4320 bytes, 715304 total

Is there any way to check why so may GC are happing and Fix this high CPU usage load issue.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.