Heap filled up after upgrade to 8.17

Hi,

I have upgraded my cluster from 8.11 to 8.17 and since it keeps crashing due to full heap used. I have upped the heap memory but it did not solve the issue. After some time with heap memory use steady, it starts growing up until using all the heap. Only a restart solves for a few days.



I keep getting these logs for failed GC:

[2025-01-08T05:08:11,647][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] memory usage down after [0], before [12256000632], after [11686031128]
[2025-01-08T05:08:11,648][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] memory usage down after [2], before [12256000632], after [11686031128]
[2025-01-08T05:08:11,649][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] memory usage down after [3], before [12256000632], after [11686031128]
[2025-01-08T05:08:11,657][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] memory usage down after [11], before [12256000632], after [11686031128]
[2025-01-08T05:08:11,659][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] memory usage down after [13], before [12306332280], after [11694419736]
[2025-01-08T05:08:11,661][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] memory usage down after [15], before [12323109496], after [11694419736]
[2025-01-08T05:08:11,667][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] memory usage down after [21], before [12365052536], after [11694419736]
[2025-01-08T05:08:11,661][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] memory usage down after [15], before [12339886712], after [11694419736]
[2025-01-08T05:08:11,663][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] memory usage down after [17], before [12356663928], after [11694419736]
[2025-01-08T05:08:12,960][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] attempting to trigger G1GC due to high heap usage [12256456472]
[2025-01-08T05:08:13,019][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] GC did bring memory usage down, before [12256456472], after [12253451072], allocations [68], duration [58]
[2025-01-08T05:08:13,019][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] memory usage down after [0], before [12441005848], after [12253451072]
[2025-01-08T05:08:13,031][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] attempt to trigger young GC failed to bring memory down, triggering full GC
[2025-01-08T05:08:18,249][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] memory usage down after [13], before [12253451072], after [11553765528]
[2025-01-08T05:08:18,252][WARN ][o.e.t.ThreadPool         ] [node-elk3] timer thread slept for [5.3s/5371ms] on absolute clock which is above the warn threshold of [5000ms]
[2025-01-08T05:08:18,252][WARN ][o.e.t.ThreadPool         ] [node-elk3] timer thread slept for [5.3s/5371003740ns] on relative clock which is above the warn threshold of [5000ms]
[2025-01-08T05:08:18,257][INFO ][o.e.m.j.JvmGcMonitorService] [node-elk3] [gc][old][182767][52] duration [5.2s], collections [1]/[5.6s], total [5.2s]/[4.3m], memory [11.2gb]->[10.7gb]/[12gb], all_pools {[CodeHeap 'non-nmethods'] [2.7mb]->[2.7mb]/[5.5mb]}{[Metaspace] [208mb]->[208mb]/[0b]}{[CodeHeap 'profiled nmethods'] [7.1mb]->[7.1mb]/[117.2mb]}{[Compressed Class Space] [28.1mb]->[28.1mb]/[1gb]}{[young] [344mb]->[8mb]/[0b]}{[old] [10.8gb]->[10.7gb]/[12gb]}{[survivor] [39.7mb]->[0b]/[0b]}{[CodeHeap 'non-profiled nmethods'] [48mb]->[48mb]/[117.2mb]}
[2025-01-08T05:08:18,257][WARN ][o.e.m.j.JvmGcMonitorService] [node-elk3] [gc][182767] overhead, spent [5.2s] collecting in the last [5.6s]
[2025-01-08T05:53:06,124][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] attempting to trigger G1GC due to high heap usage [12247381632]
[2025-01-08T05:53:06,218][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] attempt to trigger young GC failed to bring memory down, triggering full GC
[2025-01-08T05:53:11,424][WARN ][o.e.t.ThreadPool         ] [node-elk3] timer thread slept for [5.3s/5314ms] on absolute clock which is above the warn threshold of [5000ms]
[2025-01-08T05:53:11,424][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] GC did bring memory usage down, before [12247381632], after [11668240352], allocations [67], duration [5300]
[2025-01-08T05:53:11,424][WARN ][o.e.t.ThreadPool         ] [node-elk3] timer thread slept for [5.3s/5314173953ns] on relative clock which is above the warn threshold of [5000ms]
[2025-01-08T05:53:11,425][INFO ][o.e.m.j.JvmGcMonitorService] [node-elk3] [gc][old][184943][168] duration [5.1s], collections [1]/[5.9s], total [5.1s]/[13.9m], memory [11.1gb]->[10.8gb]/[12gb], all_pools {[CodeHeap 'non-nmethods'] [2.7mb]->[2.7mb]/[5.5mb]}{[Metaspace] [208.5mb]->[208.5mb]/[0b]}{[CodeHeap 'profiled nmethods'] [7mb]->[7mb]/[117.2mb]}{[Compressed Class Space] [28.2mb]->[28.2mb]/[1gb]}{[young] [304mb]->[0b]/[0b]}{[old] [10.8gb]->[10.8gb]/[12gb]}{[survivor] [19.6mb]->[0b]/[0b]}{[CodeHeap 'non-profiled nmethods'] [48.7mb]->[48.7mb]/[117.2mb]}
[2025-01-08T05:53:11,425][WARN ][o.e.m.j.JvmGcMonitorService] [node-elk3] [gc][184943] overhead, spent [5.2s] collecting in the last [5.9s]
[2025-01-08T05:53:11,425][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] could not acquire lock within 500ms when attempting to trigger G1GC due to high heap usage
[2025-01-08T05:53:11,425][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] memory usage down after [0], before [12389987968], after [11668240352]
[2025-01-08T05:53:11,426][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] could not acquire lock within 500ms when attempting to trigger G1GC due to high heap usage
[2025-01-08T05:53:11,426][INFO ][o.e.i.b.HierarchyCircuitBreakerService] [node-elk3] memory usage down after [0], before [12322879104], after [11668240352]

It does not seem to be the same issue than the one described in OOM since 8.16.1 with openjdk23 as I checked

sudo sysctl -a | grep max_map_count
vm.max_map_count = 262144

sudo wc -l /proc/374416/maps
19280 /proc/374416/maps

Here is the jvm.options config:

sudo grep -v '^#' /etc/elasticsearch/jvm.options






-XX:+UseG1GC

-Djava.io.tmpdir=${ES_TMPDIR}

20-:--add-modules=jdk.incubator.vector

23:-XX:CompileCommand=dontinline,java/lang/invoke/MethodHandle.setAsTypeCache
23:-XX:CompileCommand=dontinline,java/lang/invoke/MethodHandle.asTypeUncached


-XX:+HeapDumpOnOutOfMemoryError

-XX:+ExitOnOutOfMemoryError

-XX:HeapDumpPath=/var/lib/elasticsearch

-XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log

-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m

It was using 7g before the upgrade and was stable, now it is set to 12g on both data nodes, the third master node is not a data node and is stable.

sudo grep -v '^\s*#' /etc/elasticsearch/jvm.options.d/jvm.options
-Xms12g
-Xmx12g
GET _nodes/_all/jvm

{
  "_nodes": {
    "total": 3,
    "successful": 3,
    "failed": 0
  },
  "cluster_name": "redacted",
  "nodes": {
    "xxxxxxx-xxxxxxx": {
      "name": "elk-1",
      "transport_address": "[xxxx:xxxx:xxxx:xxxx::xxxx]:9300",
      "host": "xxxx:xxxx:xxxx:xxxx::xxxx%eth0",
      "ip": "xxxx:xxxx:xxxx:xxxx::xxxx",
      "version": "8.17.0",
      "transport_version": 8797002,
      "index_version": 8521000,
      "component_versions": {
        "transform_config_version": 10000099,
        "ml_config_version": 12000099,
        "api_key_version": 8150099
      },
      "build_flavor": "default",
      "build_type": "deb",
      "build_hash": "2b6a7fed44faa321997703718f07ee0420804b41",
      "roles": [
        "data",
        "data_cold",
        "data_content",
        "data_frozen",
        "data_hot",
        "data_warm",
        "ingest",
        "master",
        "ml",
        "remote_cluster_client",
        "transform"
      ],
      "attributes": {
        "ml.allocated_processors_double": "8.0",
        "ml.allocated_processors": "8",
        "ml.machine_memory": "25199276032",
        "transform.config_version": "10.0.0",
        "xpack.installed": "true",
        "ml.config_version": "12.0.0",
        "ml.max_jvm_size": "12884901888"
      },
      "jvm": {
        "pid": 262863,
        "version": "23",
        "vm_name": "OpenJDK 64-Bit Server VM",
        "vm_version": "23+37-2369",
        "vm_vendor": "Oracle Corporation",
        "using_bundled_jdk": true,
        "start_time_in_millis": 1736146349777,
        "mem": {
          "heap_init_in_bytes": 12884901888,
          "heap_max_in_bytes": 12884901888,
          "non_heap_init_in_bytes": 7667712,
          "non_heap_max_in_bytes": 0,
          "direct_max_in_bytes": 0
        },
        "gc_collectors": [
          "G1 Young Generation",
          "G1 Concurrent GC",
          "G1 Old Generation"
        ],
        "memory_pools": [
          "CodeHeap 'non-nmethods'",
          "Metaspace",
          "CodeHeap 'profiled nmethods'",
          "Compressed Class Space",
          "G1 Eden Space",
          "G1 Old Gen",
          "G1 Survivor Space",
          "CodeHeap 'non-profiled nmethods'"
        ],
        "using_compressed_ordinary_object_pointers": "true",
        "input_arguments": [
          "-Des.networkaddress.cache.ttl=60",
          "-Des.networkaddress.cache.negative.ttl=10",
          "-XX:+AlwaysPreTouch",
          "-Xss1m",
          "-Djava.awt.headless=true",
          "-Dfile.encoding=UTF-8",
          "-Djna.nosys=true",
          "-XX:-OmitStackTraceInFastThrow",
          "-Dio.netty.noUnsafe=true",
          "-Dio.netty.noKeySetOptimization=true",
          "-Dio.netty.recycler.maxCapacityPerThread=0",
          "-Dlog4j.shutdownHookEnabled=false",
          "-Dlog4j2.disable.jmx=true",
          "-Dlog4j2.formatMsgNoLookups=true",
          "-Djava.locale.providers=CLDR",
          "-Des.distribution.type=deb",
          "--enable-native-access=org.elasticsearch.nativeaccess,org.apache.lucene.core",
          "-XX:ReplayDataFile=/var/log/elasticsearch/replay_pid%p.log",
          "-Djava.security.manager=allow",
          "-XX:+UseG1GC",
          "-Djava.io.tmpdir=/tmp/elasticsearch-1692151378270143613",
          "--add-modules=jdk.incubator.vector",
          "-XX:CompileCommand=dontinline,java/lang/invoke/MethodHandle.setAsTypeCache",
          "-XX:CompileCommand=dontinline,java/lang/invoke/MethodHandle.asTypeUncached",
          "-XX:+HeapDumpOnOutOfMemoryError",
          "-XX:+ExitOnOutOfMemoryError",
          "-XX:HeapDumpPath=/var/lib/elasticsearch",
          "-XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log",
          "-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m",
          "-Xms12g",
          "-Xmx12g",
          "-XX:MaxDirectMemorySize=6442450944",
          "-XX:InitiatingHeapOccupancyPercent=30",
          "-XX:G1ReservePercent=25",
          "--module-path=/usr/share/elasticsearch/lib",
          "--add-modules=jdk.net",
          "--add-modules=ALL-MODULE-PATH",
          "-Djdk.module.main=org.elasticsearch.server"
        ]
      }
    },
    "xxxxxxxx_xxxxxxxx": {
      "name": "es2",
      "transport_address": "[xxxx:xxxx:xxxx:xxxx::xxxx]:9300",
      "host": "xxxx:xxxx:xxxx:xxxx::xxxx%eth0",
      "ip": "xxxx:xxxx:xxxx:xxxx::xxxx",
      "version": "8.17.0",
      "transport_version": 8797002,
      "index_version": 8521000,
      "component_versions": {
        "api_key_version": 8150099,
        "transform_config_version": 10000099,
        "ml_config_version": 12000099
      },
      "build_flavor": "default",
      "build_type": "deb",
      "build_hash": "2b6a7fed44faa321997703718f07ee0420804b41",
      "roles": [
        "master",
        "voting_only"
      ],
      "attributes": {
        "transform.config_version": "10.0.0",
        "ml.config_version": "12.0.0",
        "xpack.installed": "true"
      },
      "jvm": {
        "pid": 1247,
        "version": "23",
        "vm_name": "OpenJDK 64-Bit Server VM",
        "vm_version": "23+37-2369",
        "vm_vendor": "Oracle Corporation",
        "using_bundled_jdk": true,
        "start_time_in_millis": 1734686712271,
        "mem": {
          "heap_init_in_bytes": 1031798784,
          "heap_max_in_bytes": 1031798784,
          "non_heap_init_in_bytes": 7667712,
          "non_heap_max_in_bytes": 0,
          "direct_max_in_bytes": 0
        },
        "gc_collectors": [
          "G1 Young Generation",
          "G1 Concurrent GC",
          "G1 Old Generation"
        ],
        "memory_pools": [
          "CodeHeap 'non-nmethods'",
          "Metaspace",
          "CodeHeap 'profiled nmethods'",
          "Compressed Class Space",
          "G1 Eden Space",
          "G1 Old Gen",
          "G1 Survivor Space",
          "CodeHeap 'non-profiled nmethods'"
        ],
        "using_compressed_ordinary_object_pointers": "true",
        "input_arguments": [
          "-Des.networkaddress.cache.ttl=60",
          "-Des.networkaddress.cache.negative.ttl=10",
          "-XX:+AlwaysPreTouch",
          "-Xss1m",
          "-Djava.awt.headless=true",
          "-Dfile.encoding=UTF-8",
          "-Djna.nosys=true",
          "-XX:-OmitStackTraceInFastThrow",
          "-Dio.netty.noUnsafe=true",
          "-Dio.netty.noKeySetOptimization=true",
          "-Dio.netty.recycler.maxCapacityPerThread=0",
          "-Dlog4j.shutdownHookEnabled=false",
          "-Dlog4j2.disable.jmx=true",
          "-Dlog4j2.formatMsgNoLookups=true",
          "-Djava.locale.providers=CLDR",
          "-Des.distribution.type=deb",
          "--enable-native-access=org.elasticsearch.nativeaccess,org.apache.lucene.core",
          "-XX:ReplayDataFile=/var/log/elasticsearch/replay_pid%p.log",
          "-Djava.security.manager=allow",
          "-XX:+UseG1GC",
          "-Djava.io.tmpdir=/tmp/elasticsearch-17403265704748004512",
          "--add-modules=jdk.incubator.vector",
          "-XX:CompileCommand=dontinline,java/lang/invoke/MethodHandle.setAsTypeCache",
          "-XX:CompileCommand=dontinline,java/lang/invoke/MethodHandle.asTypeUncached",
          "-XX:+HeapDumpOnOutOfMemoryError",
          "-XX:+ExitOnOutOfMemoryError",
          "-XX:HeapDumpPath=/var/lib/elasticsearch",
          "-XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log",
          "-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m",
          "-Xms983m",
          "-Xmx983m",
          "-XX:MaxDirectMemorySize=515899392",
          "-XX:G1HeapRegionSize=4m",
          "-XX:InitiatingHeapOccupancyPercent=30",
          "-XX:G1ReservePercent=15",
          "--module-path=/usr/share/elasticsearch/lib",
          "--add-modules=jdk.net",
          "--add-modules=ALL-MODULE-PATH",
          "-Djdk.module.main=org.elasticsearch.server"
        ]
      }
    },
    "xxxxxxxxx_xxxxxxxx": {
      "name": "node-elk3",
      "transport_address": "[xxxx:xxxx:xxxx:xxxx::xxxx]:9300",
      "host": "xxxx:xxxx:xxxx:xxxx::xxxx%ens18",
      "ip": "xxxx:xxxx:xxxx:xxxx::xxxx",
      "version": "8.17.0",
      "transport_version": 8797002,
      "index_version": 8521000,
      "component_versions": {
        "ml_config_version": 12000099,
        "transform_config_version": 10000099,
        "api_key_version": 8150099
      },
      "build_flavor": "default",
      "build_type": "deb",
      "build_hash": "2b6a7fed44faa321997703718f07ee0420804b41",
      "roles": [
        "data",
        "data_cold",
        "data_content",
        "data_frozen",
        "data_hot",
        "data_warm",
        "ingest",
        "master",
        "ml",
        "remote_cluster_client",
        "transform"
      ],
      "attributes": {
        "ml.allocated_processors_double": "8.0",
        "ml.allocated_processors": "8",
        "ml.machine_memory": "25199243264",
        "transform.config_version": "10.0.0",
        "xpack.installed": "true",
        "ml.config_version": "12.0.0",
        "ml.max_jvm_size": "12884901888"
      },
      "jvm": {
        "pid": 468492,
        "version": "23",
        "vm_name": "OpenJDK 64-Bit Server VM",
        "vm_version": "23+37-2369",
        "vm_vendor": "Oracle Corporation",
        "using_bundled_jdk": true,
        "start_time_in_millis": 1736336395708,
        "mem": {
          "heap_init_in_bytes": 12884901888,
          "heap_max_in_bytes": 12884901888,
          "non_heap_init_in_bytes": 7667712,
          "non_heap_max_in_bytes": 0,
          "direct_max_in_bytes": 0
        },
        "gc_collectors": [
          "G1 Young Generation",
          "G1 Concurrent GC",
          "G1 Old Generation"
        ],
        "memory_pools": [
          "CodeHeap 'non-nmethods'",
          "Metaspace",
          "CodeHeap 'profiled nmethods'",
          "Compressed Class Space",
          "G1 Eden Space",
          "G1 Old Gen",
          "G1 Survivor Space",
          "CodeHeap 'non-profiled nmethods'"
        ],
        "using_compressed_ordinary_object_pointers": "true",
        "input_arguments": [
          "-Des.networkaddress.cache.ttl=60",
          "-Des.networkaddress.cache.negative.ttl=10",
          "-XX:+AlwaysPreTouch",
          "-Xss1m",
          "-Djava.awt.headless=true",
          "-Dfile.encoding=UTF-8",
          "-Djna.nosys=true",
          "-XX:-OmitStackTraceInFastThrow",
          "-Dio.netty.noUnsafe=true",
          "-Dio.netty.noKeySetOptimization=true",
          "-Dio.netty.recycler.maxCapacityPerThread=0",
          "-Dlog4j.shutdownHookEnabled=false",
          "-Dlog4j2.disable.jmx=true",
          "-Dlog4j2.formatMsgNoLookups=true",
          "-Djava.locale.providers=CLDR",
          "-Des.distribution.type=deb",
          "--enable-native-access=org.elasticsearch.nativeaccess,org.apache.lucene.core",
          "-XX:ReplayDataFile=/var/log/elasticsearch/replay_pid%p.log",
          "-Djava.security.manager=allow",
          "-XX:+UseG1GC",
          "-Djava.io.tmpdir=/tmp/elasticsearch-1828706303470085143",
          "--add-modules=jdk.incubator.vector",
          "-XX:CompileCommand=dontinline,java/lang/invoke/MethodHandle.setAsTypeCache",
          "-XX:CompileCommand=dontinline,java/lang/invoke/MethodHandle.asTypeUncached",
          "-XX:+HeapDumpOnOutOfMemoryError",
          "-XX:+ExitOnOutOfMemoryError",
          "-XX:HeapDumpPath=/var/lib/elasticsearch",
          "-XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log",
          "-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m",
          "-Xms12g",
          "-Xmx12g",
          "-XX:MaxDirectMemorySize=6442450944",
          "-XX:InitiatingHeapOccupancyPercent=30",
          "-XX:G1ReservePercent=25",
          "--module-path=/usr/share/elasticsearch/lib",
          "--add-modules=jdk.net",
          "--add-modules=ALL-MODULE-PATH",
          "-Djdk.module.main=org.elasticsearch.server"
        ]
      }
    }
  }
}

Servers are on Ubuntu 24.04 and Elasticsearch is using bundled Java

{
  "_nodes": {
    "total": 3,
    "successful": 3,
    "failed": 0
  },
  "cluster_name": "redacted",
  "cluster_uuid": "some_uuid",
  "timestamp": 1736332710536,
  "status": "green",
  "indices": {
    "count": 695,
    "shards": {
      "total": 1418,
      "primaries": 709,
      "replication": 1,
      "index": {
        "shards": {
          "min": 2,
          "max": 6,
          "avg": 2.0402877697841726
        },
        "primaries": {
          "min": 1,
          "max": 3,
          "avg": 1.0201438848920863
        },
        "replication": {
          "min": 1,
          "max": 1,
          "avg": 1
        }
      }
    },
    "docs": {
      "count": 1152459769,
      "deleted": 402324,
      "total_size": "354.9gb",
      "total_size_in_bytes": 381118982398
    },
    "store": {
      "size": "713.8gb",
      "size_in_bytes": 766525582487,
      "total_data_set_size": "713.8gb",
      "total_data_set_size_in_bytes": 766525582487,
      "reserved": "0b",
      "reserved_in_bytes": 0
    },
    "fielddata": {
      "memory_size": "414.6kb",
      "memory_size_in_bytes": 424552,
      "evictions": 905,
      "global_ordinals": {
        "build_time": "1.7m",
        "build_time_in_millis": 103310
      }
    },
    "query_cache": {
      "memory_size": "2.3gb",
      "memory_size_in_bytes": 2576805750,
      "total_count": 10094819,
      "hit_count": 3144757,
      "miss_count": 6950062,
      "cache_size": 145463,
      "cache_count": 235321,
      "evictions": 89858
    },
    "completion": {
      "size": "0b",
      "size_in_bytes": 0
    },
    "segments": {
      "count": 34124,
      "memory": "0b",
      "memory_in_bytes": 0,
      "terms_memory": "0b",
      "terms_memory_in_bytes": 0,
      "stored_fields_memory": "0b",
      "stored_fields_memory_in_bytes": 0,
      "term_vectors_memory": "0b",
      "term_vectors_memory_in_bytes": 0,
      "norms_memory": "0b",
      "norms_memory_in_bytes": 0,
      "points_memory": "0b",
      "points_memory_in_bytes": 0,
      "doc_values_memory": "0b",
      "doc_values_memory_in_bytes": 0,
      "index_writer_memory": "108.5mb",
      "index_writer_memory_in_bytes": 113788976,
      "version_map_memory": "226.7kb",
      "version_map_memory_in_bytes": 232207,
      "fixed_bit_set": "7.7mb",
      "fixed_bit_set_memory_in_bytes": 8108384,
      "max_unsafe_auto_id_timestamp": 1736330338381,
      "file_sizes": {}
    },
    "mappings": {
      "total_field_count": 769680,
      "total_deduplicated_field_count": 100847,
      "total_deduplicated_mapping_size": "725.7kb",
      "total_deduplicated_mapping_size_in_bytes": 743159,
      "field_types": [
        {
          "name": "aggregate_metric_double",
          "count": 202,
          "index_count": 101,
          "script_count": 0
        },
        {
          "name": "alias",
          "count": 2724,
          "index_count": 139,
          "script_count": 0
        },
        {
          "name": "binary",
          "count": 3,
          "index_count": 3,
          "script_count": 0
        },
        {
          "name": "boolean",
          "count": 15155,
          "index_count": 403,
          "script_count": 0
        },
        {
          "name": "byte",
          "count": 98,
          "index_count": 98,
          "script_count": 0
        },
        {
          "name": "constant_keyword",
          "count": 2142,
          "index_count": 490,
          "script_count": 0
        },
        {
          "name": "counted_keyword",
          "count": 1,
          "index_count": 1,
          "script_count": 0
        },
        {
          "name": "date",
          "count": 21153,
          "index_count": 642,
          "script_count": 0
        },
        {
          "name": "date_nanos",
          "count": 1,
          "index_count": 1,
          "script_count": 0
        },
        {
          "name": "date_range",
          "count": 13,
          "index_count": 13,
          "script_count": 0
        },
        {
          "name": "double",
          "count": 3479,
          "index_count": 132,
          "script_count": 0
        },
        {
          "name": "double_range",
          "count": 1,
          "index_count": 1,
          "script_count": 0
        },
        {
          "name": "flattened",
          "count": 4265,
          "index_count": 126,
          "script_count": 0
        },
        {
          "name": "float",
          "count": 4166,
          "index_count": 228,
          "script_count": 0
        },
        {
          "name": "float_range",
          "count": 1,
          "index_count": 1,
          "script_count": 0
        },
        {
          "name": "geo_point",
          "count": 1345,
          "index_count": 207,
          "script_count": 0
        },
        {
          "name": "geo_shape",
          "count": 1,
          "index_count": 1,
          "script_count": 0
        },
        {
          "name": "half_float",
          "count": 87,
          "index_count": 28,
          "script_count": 0
        },
        {
          "name": "histogram",
          "count": 109,
          "index_count": 109,
          "script_count": 0
        },
        {
          "name": "integer",
          "count": 584,
          "index_count": 79,
          "script_count": 0
        },
        {
          "name": "integer_range",
          "count": 1,
          "index_count": 1,
          "script_count": 0
        },
        {
          "name": "ip",
          "count": 13221,
          "index_count": 350,
          "script_count": 0
        },
        {
          "name": "ip_range",
          "count": 91,
          "index_count": 91,
          "script_count": 0
        },
        {
          "name": "keyword",
          "count": 455180,
          "index_count": 645,
          "script_count": 1,
          "lang": [
            "painless"
          ],
          "lines_max": 15,
          "lines_total": 15,
          "chars_max": 522,
          "chars_total": 522,
          "source_max": 0,
          "source_total": 0,
          "doc_max": 0,
          "doc_total": 0
        },
        {
          "name": "long",
          "count": 112223,
          "index_count": 597,
          "script_count": 0
        },
        {
          "name": "long_range",
          "count": 1,
          "index_count": 1,
          "script_count": 0
        },
        {
          "name": "match_only_text",
          "count": 7466,
          "index_count": 334,
          "script_count": 0
        },
        {
          "name": "nested",
          "count": 1801,
          "index_count": 164,
          "script_count": 0
        },
        {
          "name": "object",
          "count": 101675,
          "index_count": 641,
          "script_count": 0
        },
        {
          "name": "rank_features",
          "count": 1,
          "index_count": 1,
          "script_count": 0
        },
        {
          "name": "scaled_float",
          "count": 364,
          "index_count": 150,
          "script_count": 0
        },
        {
          "name": "semantic_text",
          "count": 1,
          "index_count": 1,
          "script_count": 0
        },
        {
          "name": "shape",
          "count": 1,
          "index_count": 1,
          "script_count": 0
        },
        {
          "name": "short",
          "count": 9292,
          "index_count": 98,
          "script_count": 0
        },
        {
          "name": "text",
          "count": 10872,
          "index_count": 264,
          "script_count": 0
        },
        {
          "name": "unsigned_long",
          "count": 96,
          "index_count": 14,
          "script_count": 0
        },
        {
          "name": "version",
          "count": 28,
          "index_count": 28,
          "script_count": 0
        },
        {
          "name": "wildcard",
          "count": 1836,
          "index_count": 144,
          "script_count": 0
        }
      ],
      "runtime_field_types": [],
      "source_modes": {
        "stored": 647
      }
    },
    "analysis": {
      "char_filter_types": [],
      "tokenizer_types": [],
      "filter_types": [],
      "analyzer_types": [],
      "built_in_char_filters": [],
      "built_in_tokenizers": [],
      "built_in_filters": [],
      "built_in_analyzers": [],
      "synonyms": {}
    },
    "versions": [
      {
        "version": "7.6.1",
        "index_count": 4,
        "primary_shard_count": 4,
        "total_primary_size": "2.3mb",
        "total_primary_bytes": 2486588
      },
      {
        "version": "7.8.0",
        "index_count": 5,
        "primary_shard_count": 5,
        "total_primary_size": "2.9mb",
        "total_primary_bytes": 3119813
      },
      {
        "version": "7.9.2",
        "index_count": 1,
        "primary_shard_count": 1,
        "total_primary_size": "12.5mb",
        "total_primary_bytes": 13126699
      },
      {
        "version": "7.10.0",
        "index_count": 2,
        "primary_shard_count": 2,
        "total_primary_size": "12.8mb",
        "total_primary_bytes": 13481193
      },
      {
        "version": "7.14.1",
        "index_count": 7,
        "primary_shard_count": 7,
        "total_primary_size": "6.2gb",
        "total_primary_bytes": 6727807089
      },
      {
        "version": "7.16.0",
        "index_count": 51,
        "primary_shard_count": 51,
        "total_primary_size": "9.3gb",
        "total_primary_bytes": 10068147476
      },
      {
        "version": "7.17.9",
        "index_count": 40,
        "primary_shard_count": 40,
        "total_primary_size": "56.4mb",
        "total_primary_bytes": 59185531
      },
      {
        "version": "8.11.0-8.11.4",
        "index_count": 313,
        "primary_shard_count": 313,
        "total_primary_size": "30.3gb",
        "total_primary_bytes": 32591243631
      },
      {
        "version": "8.17.0",
        "index_count": 272,
        "primary_shard_count": 286,
        "total_primary_size": "309.7gb",
        "total_primary_bytes": 332593709921
      }
    ],
    "search": {
      "total": 1947871,
      "queries": {
        "bool": 1858961,
        "prefix": 79,
        "match": 121870,
        "range": 1703200,
        "nested": 3086,
        "wildcard": 2,
        "match_phrase": 46896,
        "terms": 1586376,
        "match_all": 2,
        "match_phrase_prefix": 6,
        "exists": 1663023,
        "term": 1852825,
        "simple_query_string": 58060
      },
      "rescorers": {},
      "sections": {
        "highlight": 17,
        "search_after": 15237,
        "stored_fields": 34,
        "runtime_mappings": 78286,
        "query": 1858973,
        "script_fields": 34,
        "terminate_after": 11,
        "pit": 37107,
        "_source": 9758,
        "fields": 4184,
        "collapse": 36594,
        "aggs": 230989
      },
      "retrievers": {}
    },
    "dense_vector": {
      "value_count": 0
    },
    "sparse_vector": {
      "value_count": 0
    }
  },
  "nodes": {
    "count": {
      "total": 3,
      "coordinating_only": 0,
      "data": 2,
      "data_cold": 2,
      "data_content": 2,
      "data_frozen": 2,
      "data_hot": 2,
      "data_warm": 2,
      "index": 0,
      "ingest": 2,
      "master": 3,
      "ml": 2,
      "remote_cluster_client": 2,
      "search": 0,
      "transform": 2,
      "voting_only": 1
    },
    "versions": [
      "8.17.0"
    ],
    "os": {
      "available_processors": 18,
      "allocated_processors": 18,
      "names": [
        {
          "name": "Linux",
          "count": 3
        }
      ],
      "pretty_names": [
        {
          "pretty_name": "Ubuntu 24.04.1 LTS",
          "count": 3
        }
      ],
      "architectures": [
        {
          "arch": "amd64",
          "count": 3
        }
      ],
      "mem": {
        "total": "48.8gb",
        "total_in_bytes": 52461871104,
        "adjusted_total": "48.8gb",
        "adjusted_total_in_bytes": 52461871104,
        "free": "1gb",
        "free_in_bytes": 1087062016,
        "used": "47.8gb",
        "used_in_bytes": 51374809088,
        "free_percent": 2,
        "used_percent": 98
      }
    },
    "process": {
      "cpu": {
        "percent": 53
      },
      "open_file_descriptors": {
        "min": 629,
        "max": 3751,
        "avg": 2708
      }
    },
    "jvm": {
      "max_uptime": "19d",
      "max_uptime_in_millis": 1645999034,
      "versions": [
        {
          "version": "23",
          "vm_name": "OpenJDK 64-Bit Server VM",
          "vm_version": "23+37-2369",
          "vm_vendor": "Oracle Corporation",
          "bundled_jdk": true,
          "using_bundled_jdk": true,
          "count": 3
        }
      ],
      "mem": {
        "heap_used": "20.5gb",
        "heap_used_in_bytes": 22104554928,
        "heap_max": "24.9gb",
        "heap_max_in_bytes": 26801602560
      },
      "threads": 457
    },
    "fs": {
      "total": "1.5tb",
      "total_in_bytes": 1707082899456,
      "free": "755.5gb",
      "free_in_bytes": 811279769600,
      "available": "720.7gb",
      "available_in_bytes": 773864079360
    },
    "plugins": [],
    "network_types": {
      "transport_types": {
        "security4": 3
      },
      "http_types": {
        "security4": 3
      }
    },
    "discovery_types": {
      "multi-node": 3
    },
    "packaging_types": [
      {
        "flavor": "default",
        "type": "deb",
        "count": 3
      }
    ],
    "ingest": {
      "number_of_pipelines": 305,
      "processor_stats": {
        "append": {
          "count": 1663342,
          "failed": 0,
          "current": 0,
          "time": "6.5s",
          "time_in_millis": 6563
        },
        "attachment": {
          "count": 0,
          "failed": 0,
          "current": 0,
          "time": "0s",
          "time_in_millis": 0
        },
        "community_id": {
          "count": 0,
          "failed": 0,
          "current": 0,
          "time": "0s",
          "time_in_millis": 0
        },
        "convert": {
          "count": 1263266,
          "failed": 0,
          "current": 0,
          "time": "2.3s",
          "time_in_millis": 2305
        },
        "csv": {
          "count": 0,
          "failed": 0,
          "current": 0,
          "time": "0s",
          "time_in_millis": 0
        },
        "date": {
          "count": 7646722,
          "failed": 0,
          "current": 0,
          "time": "1.6m",
          "time_in_millis": 98818
        },
        "date_index_name": {
          "count": 0,
          "failed": 0,
          "current": 0,
          "time": "0s",
          "time_in_millis": 0
        },
        "dissect": {
          "count": 0,
          "failed": 0,
          "current": 0,
          "time": "0s",
          "time_in_millis": 0
        },
        "dot_expander": {
          "count": 66944,
          "failed": 0,
          "current": 0,
          "time": "1.7s",
          "time_in_millis": 1756
        },
        "drop": {
          "count": 804884,
          "failed": 0,
          "current": 0,
          "time": "120ms",
          "time_in_millis": 120
        },
        "enrich": {
          "count": 217075328,
          "failed": 3376,
          "current": 0,
          "time": "1.3d",
          "time_in_millis": 114007477
        },
        "fail": {
          "count": 0,
          "failed": 0,
          "current": 0,
          "time": "0s",
          "time_in_millis": 0
        },
        "fingerprint": {
          "count": 12584063,
          "failed": 0,
          "current": 0,
          "time": "42.9s",
          "time_in_millis": 42994
        },
        "foreach": {
          "count": 14279,
          "failed": 0,
          "current": 0,
          "time": "206ms",
          "time_in_millis": 206
        },
        "geoip": {
          "count": 240094323,
          "failed": 0,
          "current": 0,
          "time": "28.7m",
          "time_in_millis": 1726387
        },
        "grok": {
          "count": 10730001,
          "failed": 642745,
          "current": 0,
          "time": "7.5m",
          "time_in_millis": 450424
        },
        "gsub": {
          "count": 0,
          "failed": 0,
          "current": 0,
          "time": "0s",
          "time_in_millis": 0
        },
        "inference": {
          "count": 0,
          "failed": 0,
          "current": 0,
          "time": "0s",
          "time_in_millis": 0
        },
        "join": {
          "count": 0,
          "failed": 0,
          "current": 0,
          "time": "0s",
          "time_in_millis": 0
        },
        "json": {
          "count": 66944,
          "failed": 0,
          "current": 0,
          "time": "1.6s",
          "time_in_millis": 1613
        },
        "kv": {
          "count": 0,
          "failed": 0,
          "current": 0,
          "time": "0s",
          "time_in_millis": 0
        },
        "lowercase": {
          "count": 0,
          "failed": 0,
          "current": 0,
          "time": "0s",
          "time_in_millis": 0
        },
        "pipeline": {
          "count": 403040954,
          "failed": 0,
          "current": 0,
          "time": "3.6m",
          "time_in_millis": 217691
        },
        "remove": {
          "count": 312415699,
          "failed": 0,
          "current": 0,
          "time": "3.8m",
          "time_in_millis": 228604
        },
        "rename": {
          "count": 253835892,
          "failed": 1049908,
          "current": 0,
          "time": "13m",
          "time_in_millis": 782411
        },
        "reroute": {
          "count": 0,
          "failed": 0,
          "current": 0,
          "time": "0s",
          "time_in_millis": 0
        },
        "script": {
          "count": 196616849,
          "failed": 0,
          "current": 0,
          "time": "5.5m",
          "time_in_millis": 330215
        },
        "set": {
          "count": 22788381,
          "failed": 0,
          "current": 0,
          "time": "1m",
          "time_in_millis": 60294
        },
        "set_security_user": {
          "count": 96265925,
          "failed": 0,
          "current": 0,
          "time": "7.9m",
          "time_in_millis": 476935
        },
        "split": {
          "count": 2514457,
          "failed": 939378,
          "current": 0,
          "time": "11.6s",
          "time_in_millis": 11667
        },
        "trim": {
          "count": 0,
          "failed": 0,
          "current": 0,
          "time": "0s",
          "time_in_millis": 0
        },
        "uppercase": {
          "count": 0,
          "failed": 0,
          "current": 0,
          "time": "0s",
          "time_in_millis": 0
        },
        "uri_parts": {
          "count": 482761,
          "failed": 1400,
          "current": 0,
          "time": "2.8s",
          "time_in_millis": 2881
        },
        "user_agent": {
          "count": 3214860,
          "failed": 0,
          "current": 0,
          "time": "13.6s",
          "time_in_millis": 13673
        }
      }
    },
    "indexing_pressure": {
      "memory": {
        "current": {
          "combined_coordinating_and_primary": "0b",
          "combined_coordinating_and_primary_in_bytes": 0,
          "coordinating": "0b",
          "coordinating_in_bytes": 0,
          "primary": "0b",
          "primary_in_bytes": 0,
          "replica": "0b",
          "replica_in_bytes": 0,
          "all": "0b",
          "all_in_bytes": 0
        },
        "total": {
          "combined_coordinating_and_primary": "0b",
          "combined_coordinating_and_primary_in_bytes": 0,
          "coordinating": "0b",
          "coordinating_in_bytes": 0,
          "primary": "0b",
          "primary_in_bytes": 0,
          "replica": "0b",
          "replica_in_bytes": 0,
          "all": "0b",
          "all_in_bytes": 0,
          "coordinating_rejections": 0,
          "primary_rejections": 0,
          "replica_rejections": 0,
          "primary_document_rejections": 0
        },
        "limit": "0b",
        "limit_in_bytes": 0
      }
    }
  },
  "snapshots": {
    "current_counts": {
      "snapshots": 0,
      "shard_snapshots": 0,
      "snapshot_deletions": 0,
      "concurrent_operations": 0,
      "cleanups": 0
    },
    "repositories": {
      "s3-ovh": {
        "type": "s3",
        "current_counts": {
          "snapshots": 0,
          "clones": 0,
          "finalizations": 0,
          "deletions": 0,
          "snapshot_deletions": 0,
          "active_deletions": 0,
          "shards": {
            "total": 0,
            "complete": 0,
            "incomplete": 0,
            "states": {
              "INIT": 0,
              "SUCCESS": 0,
              "FAILED": 0,
              "ABORTED": 0,
              "MISSING": 0,
              "WAITING": 0,
              "QUEUED": 0,
              "PAUSED_FOR_NODE_REMOVAL": 0
            }
          }
        },
        "oldest_start_time": "2025-01-08T10:38:30.276Z",
        "oldest_start_time_millis": 1736332710276
      }
    }
  },
  "repositories": {
    "s3": {
      "count": 1,
      "read_write": 1
    }
  },
  "ccs": {
    "_search": {
      "total": 0,
      "success": 0,
      "skipped": 0,
      "took": {
        "max": 0,
        "avg": 0,
        "p90": 0
      },
      "took_mrt_true": {
        "max": 0,
        "avg": 0,
        "p90": 0
      },
      "took_mrt_false": {
        "max": 0,
        "avg": 0,
        "p90": 0
      },
      "remotes_per_search_max": 0,
      "remotes_per_search_avg": 0,
      "failure_reasons": {},
      "features": {},
      "clients": {},
      "clusters": {}
    }
  }
}

Any idea what I could check to see what could be the cause of this issue ?