Preformatted texto we run Elastic on GKE using ECK. We are on version 8.12. I just installed the Fleet server yesterday with 4 replicas and they are sitting behind a GCP classic http LB.
The servers can be seen on the fleet page and have not gone offline. CPU/RAM are well below limits.
So I added about 5 agents to test out everything and I noticed they go offine randomly and come back for some time and then offline again.
From the agent logs I see:
09:58:03.111
elastic_agent
[elastic_agent][warn] Possible transient error during checkin with fleet-server, retrying
10:00:59.077
elastic_agent
[elastic_agent][error] Cannot checkin in with fleet-server, retrying
There are quite a few of the retrying messages but it eventually connects. I see hundreds of 502 errors on the LB. It is setup exactly the way my APM server, Elastic and Kibana LBs are configured and they have no issue.
Any ideas? The error is kind of vague. I did try to set the affinity to client ip but no luck.
Thanks