i got the following in elasticsearch container logs:
log:[parent] Data too large, data for [<transport_request>] would be [1020455062/973.1mb], which is larger than the limit of [1011774259/964.9mb], real usage: [1020454800/973.1mb], new bytes reserved: [262/262b]
After a research i find that this is the solution:
modify java options via environment ES_JAVA_OPTS to increase heap size
increase both values at the same time Xms and Xmx.
I made the change and change it to 2gb instead of 1gb in both. After the change the pods failed to start with the following log:
{"type": "server", "timestamp": "2020-07-17T17:41:36,200Z", "level": "INFO", "component": "o.e.c.s.MasterService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-0", "message": "node-join[{elasticsearch-master-1}{Qh_vIzeDRlyGuFCiXMnjKg}{dLaiKIj7SIaTJryxi3bz2g}{10.42.1.98}{10.42.1.98:9300}{dilm}{ml.machine_memory=4294967296, ml.max_open_jobs=20, xpack.installed=true} join existing leader], term: 55, version: 37066, delta: added {{elasticsearch-master-1}{Qh_vIzeDRlyGuFCiXMnjKg}{dLaiKIj7SIaTJryxi3bz2g}{10.42.1.98}{10.42.1.98:9300}{dilm}{ml.machine_memory=4294967296, ml.max_open_jobs=20, xpack.installed=true}}", "cluster.uuid": "pQepCBd7RBiiN9UQHlvppA", "node.id": "IX4HsayRQzqHk1I1K6K5Lg" }
7/17/2020 8:41:45 PM {"type": "server", "timestamp": "2020-07-17T17:41:45,216Z", "level": "INFO", "component": "o.e.c.s.ClusterApplierService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-0", "message": "added {{elasticsearch-master-1}{Qh_vIzeDRlyGuFCiXMnjKg}{dLaiKIj7SIaTJryxi3bz2g}{10.42.1.98}{10.42.1.98:9300}{dilm}{ml.machine_memory=4294967296, ml.max_open_jobs=20, xpack.installed=true}}, term: 55, version: 37066, reason: Publication{term=55, version=37066}", "cluster.uuid": "pQepCBd7RBiiN9UQHlvppA", "node.id": "IX4HsayRQzqHk1I1K6K5Lg" }
Please assist.
Thanks,
Dawood