I'm using elasticsearch as database but I'm facing momentary surge in search rate and latency. Can anyone tell why there is a high search rate and latency observerd I have 10 nodes of cluster which are physical vms. It's working fine as of now but and but it keeps happening once in a whole day. I have increased the threadpool and heap memory both but didn't help.
java.lang.Thread.run(Thread.java:748)
Suppressed: org.elasticsearch.client.ResponseException: method [POST], host [http://172.24.44.41:9200], URI [/cmsentitydb_20210215/_search?routing=83&typed_keys=true&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&ignore_throttled=true&search_type=query_then_fetch&batched_reduce_size=512&ccs_minimize_roundtrips=true], status line [HTTP/1.1 429 Too Many Requests]{"error":{"root_cause":[{"type":"circuit_breaking_exception","reason":"[parent] Data too large, data for [<http_request>] would be [33441913798/31.1gb], which is larger than the limit of [32641751449/30.3gb], real usage: [33441913216/31.1gb], new bytes reserved: [582/582b], usages [request=16472/16kb, fielddata=1771/1.7kb, in_flight_requests=17221480/16.4mb, model_inference=0/0b, eql_sequence=0/0b, accounting=4393158/4.1mb]","bytes_wanted":33441913798,"bytes_limit":32641751449,"durability":"TRANSIENT"}],"type":"circuit_breaking_exception","reason":"[parent] Data too large, data for [<http_request>] would be [33441913798/31.1gb], which is larger than the limit of [32641751449/30.3gb], real usage: [33441913216/31.1gb], new bytes reserved: [582/582b], usages [request=16472/16kb, fielddata=1771/1.7kb, in_flight_requests=17221480/16.4mb, model_inference=0/0b, eql_sequence=0/0b, accounting=4393158/4.1mb]","bytes_wanted":33441913798,"bytes_limit":32641751449,"durability":"TRANSIENT"},"status":429}