Hello folks
Our clients encountered errors below recently when performing index and query requests:
TransportError(429, u'circuit_breaking_exception', {u'status': 429, u'error': {u'bytes_wanted': 32385970160, u'durability': u'PERMANENT', u'root_cause': [{u'reason': u'[parent] Data too large, data for [<http_request>] would be [32385970160/30.1gb], which is larger than the limit of [32266702028/30gb], real usage: [32385970160/30.1gb], new bytes reserved: [0/0b], usages [request=0/0b, fielddata=8258276/7.8mb, in_flight_requests=54937218/52.3mb, model_inference=0/0b, accounting=137025656/130.6mb]', u'type': u'circuit_breaking_exception', u'bytes_wanted': 32385970160, u'bytes_limit': 32266702028, u'durability': u'PERMANENT'}], u'reason': u'[parent] Data too large, data for [<http_request>] would be [32385970160/30.1gb], which is larger than the limit of [32266702028/30gb], real usage: [32385970160/30.1gb], new bytes reserved: [0/0b], usages [request=0/0b, fielddata=8258276/7.8mb, in_flight_requests=54937218/52.3mb, model_inference=0/0b, accounting=137025656/130.6mb]', u'type': u'circuit_breaking_exception', u'bytes_limit': 32266702028}}), {'\xe9\x9b\x86\xe7\xbe\xa4\xe5\x90\x8d\xe7\xa7\xb0': u'lc_qa_6', '\xe4\xb8\xa5\xe9\x87\x8d\xe7\xad\x89\xe7\xba\xa7': '\xe4\xb8\xad\xe5\xba\xa6', '\xe6\x97\xa5\xe5\xbf\x97\xe5\x90\x8d\xe7\xa7\xb0': None}
I used Eclipse memory analyzer to analyze the hprof dump and found that CompressingStoredFieldsReader retained heap size occupied more than 70% of the total heap size :
I want to know why CompressingStoredFieldsReader instances take up a lot of memory and how to solve this problem.
Thanks