ElasticSearch high CPU load and Excessive garbage collection

Elastic Stack Elasticsearch
1

Hi, I am using 3 node cluster in elastic search having configuration 4core 16GB. Heap size allocated for elastic search is 8GB. I have only 3 indices in my cluster having a total of 17lac records (in all 3 indices combined).

I don't do updates on them. Only search queries are performed on them. But I face high CPU usage often in any of the nodes. The version of elastic search, I am using, is 2.4.

Following are my GC logs:-

CommandLine flags: -XX:CMSInitiatingOccupancyFraction=75 -XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError -XX:InitialHeapSize=8589934592 -XX:MaxHeapSize=8589934592 -XX:MaxNewSize=348966912 -XX:MaxTenuringThreshold=6 -XX:NewSize=348966912 -XX:OldPLABSize=16 -XX:OldSize=697933824 -XX:+PrintClassHistogram -XX:+PrintGC -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+UseParNewGC

273655K->1054K(306688K), 0.0174304 secs] 1594728K->1322130K(8354560K), 0.0175535 secs] [Times: user=0.07 sys=0.00, real=0.02 secs]
2019-01-08T23:47:41.027+0530: 4855959.377: Total time for which application threads were stopped: 0.0183525 seconds, Stopping threads took: 0.0000959 seconds
2019-01-08T23:49:37.595+0530: 4856075.945: [GC (Allocation Failure) 2019-01-08T23:49:37.595+0530: 4856075.945: [ParNew
Desired survivor size 17432576 bytes, new threshold 6 (max 6)

  • age 1: 724392 bytes, 724392 total
  • age 2: 3920 bytes, 728312 total
  • age 3: 36176 bytes, 764488 total
  • age 4: 4344 bytes, 768832 total
  • age 5: 2400 bytes, 771232 total
  • age 6: 2720 bytes, 773952 total
    : 273694K->1055K(306688K), 0.0185617 secs] 1594770K->1322133K(8354560K), 0.0187332 secs] [Times: user=0.07 sys=0.00, real=0.02 secs]
    2019-01-08T23:49:37.614+0530: 4856075.964: Total time for which application threads were stopped: 0.0196404 seconds, Stopping threads took: 0.0001491 seconds
    2019-01-08T23:51:23.625+0530: 4856181.975: [GC (Allocation Failure) 2019-01-08T23:51:23.625+0530: 4856181.975: [ParNew
    Desired survivor size 17432576 bytes, new threshold 6 (max 6)
  • age 1: 698992 bytes, 698992 total
  • age 2: 39024 bytes, 738016 total
  • age 3: 3864 bytes, 741880 total
  • age 4: 36176 bytes, 778056 total
  • age 5: 4320 bytes, 782376 total
  • age 6: 2400 bytes, 784776 total
    : 273695K->1193K(306688K), 0.0229432 secs] 1594773K->1322273K(8354560K), 0.0230595 secs] [Times: user=0.07 sys=0.00, real=0.02 secs]
    2019-01-08T23:51:23.648+0530: 4856181.999: Total time for which application threads were stopped: 0.0239003 seconds, Stopping threads took: 0.0001023 seconds
    2019-01-08T23:53:21.817+0530: 4856300.167: [GC (Allocation Failure) 2019-01-08T23:53:21.817+0530: 4856300.167: [ParNew
    Desired survivor size 17432576 bytes, new threshold 6 (max 6)
  • age 1: 628344 bytes, 628344 total
  • age 2: 3888 bytes, 632232 total
  • age 3: 38712 bytes, 670944 total
  • age 4: 3864 bytes, 674808 total
  • age 5: 36176 bytes, 710984 total
  • age 6: 4320 bytes, 715304 total

Is there any way to check why so may GC are happing and Fix this high CPU usage load issue.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.