Monitor GC duration/count

Elastic Stack Elasticsearch
1

Hello,

I want know
how many GC are executed in last 5 min
how long duration's GC is ?

I want use : GET /_nodes/elasticsearch_instance1/stats/jvm :
Output return :

      "jvm": {
    "timestamp": 1554888379405,
    "uptime_in_millis": 56643256,
    "mem": {
      "heap_used_in_bytes": 16247615560,
      "heap_used_percent": 50,
      "heap_committed_in_bytes": 32212254720,
      "heap_max_in_bytes": 32212254720,
      "non_heap_used_in_bytes": 115543704,
      "non_heap_committed_in_bytes": 122445824,
      "pools": {
        "young": {
          "used_in_bytes": 14663286784,
          "max_in_bytes": 0,
          "peak_used_in_bytes": 18001952768,
          "peak_max_in_bytes": 0
        },
        "survivor": {
          "used_in_bytes": 109051904,
          "max_in_bytes": 0,
          "peak_used_in_bytes": 385875968,
          "peak_max_in_bytes": 0
        },
        "old": {
          "used_in_bytes": 1475276872,
          "max_in_bytes": 32212254720,
          "peak_used_in_bytes": 1475276872,
          "peak_max_in_bytes": 32212254720
        }
      }
    },
    "gc": {
      "collectors": {
        "young": {
          "collection_count": 22,
          "collection_time_in_millis": 1462
        },
        "old": {
          "collection_count": 0,
          "collection_time_in_millis": 0
        }
      }
    },
indent preformatted text by 4 spaces

So, i don't understand return values ?

My final aim is to put value is a text file to create graph about it :

exemple :

date_every_5min ; instances es ; nbre gc ; average duration gc
100419_11_30 ; elasticsearch_instance1; 10 ; 100ms
100419_11_30 ; elasticsearch_instance2; 5 ; 60ms
100419_11_35 ; elasticsearch_instance3; 7 ;  70ms
100419_11_35 ; elasticsearch_instance1; 7 ;  35ms
100419_11_35 ; elasticsearch_instance2; 3 ;  86ms
...

Is possible ?

2

For example, what represente value in count ? For the current day ? Last XX minutes ?

Same on :

          "collection_time_in_millis": 1462

This is the duration of GC young, but it is duration for the current day ? (so i must divise this value by the number of hour in the day ?)

3

The GC statistics you list are all cumulative, increasing from zero since the node was started. If you want to measure statistics over a time period then you should measure the values at the start and end of the time period and subtract them.

4

Waw it's not intuitive.

My node started 08/04/2019 at 8.00 am.
=> Currently its have 22 count and 1462 ms of duration
If i follow you, average day : 11,5 count et 731 ms of duration (so 65 ms by gc)

It's possible to use monitor metrics to create this type of visualization on Kibana ?

5

Reporting cumulative statistics like this is pretty common, see e.g. /proc/diskstats, /proc/net/dev, Postgres statistics and many more. It would be much less intuitive (and much more computationally expensive) for these numbers to represent events within some arbitrary time window.

Yes, see https://www.elastic.co/guide/en/kibana/master/timelion-math.html for an example.

6

I use free X-Pack (on 5).

In, Nodes, Advanced, i can see 2 graph very interesting :

GC Count and GC Duration on nodes. How to reproduce the same with curl API from shell.

I want that, because free X-Pack have 7 days of retention. I want store more days, so i must use log file in Linux.

7

7 days is the default for xpack.monitoring.history.duration but if you want to retain more data then you can increase this setting. The basic X-pack licence doesn't limit the amount of data you can retain as far as I can tell.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.