Elastic Agents error after installation: ...fleet-server returned an error: MaxLimit

[jamespichardo]

After installing, Elastic Agents it keeps in "Offline" state and "Healthy" intermittently. When checking Agent logs I found this error several times:

--
[elastic_agent][error] Could not communicate with fleet-server Checking API will retry, error: status code: 429, fleet-server returned an error: MaxLimit, message: exceeded the max limit

Please note that I have APM-Fleet server in Elastic cloud with 1GB RAM (2 zones). Attached you'll find a screenshot where you can see I don't even have 1,000 agents installed.

Additionally:

Cheers,
James.

[blaker (Blake Rouse)]

The default settings for the Elastic Agent in Cloud have a maximum connection limit on Fleet Server. This is because the default instance type is only 512MB so we set a default connection limit so the instance does not run out of memory. Because you are using larger instances you can increase this limit to allow more Elastic Agent's to connect at once.

The limit is set inside of the Elastic Agent on Cloud policy. Modify the fleet-server integration inside that policy to increase the max connections limit.

[mostlyjason (Jason Skowronski)]

Here is a screenshot of the page where you need to raise the connection limit. FYI there are additional limits if you expand the advanced options. We plan to have docs with more guidance on this soon.

[mostlyjason (Jason Skowronski)]

One more setting that you need to change is the max number of checkins. The max connection setting in the UI should be great than or equal to the sum of the maxes for all the server limits. So in the example below, it should be set to at least 100 + 10 + 20 + 10. This limit is enforced before TLS is evaluated, so its a way to prevent DDOS. We're working to improve this UX in an upcoming release.