APM server did not respond within 10s of gzip stream finish

Elastic Observability APM
,
1

If you are asking about a problem you are experiencing, please use the following template, as it will help us help you. If you have a different problem, please delete all of this text :slight_smile:

TIP 1: select at least one tag that further categorizes your topic. For example server for APM Server related questions, java for questions regarding the Elastic APM Java agent, or ui for questions about the APM App within Kibana.

TIP 2: Check out the troubleshooting guide first. Not only will it help you to resolve common problems faster but it also explains in more detail which information we need before we can properly help you.

Kibana version: 8.7.1

Elasticsearch version: 8.7.1

APM Server version: 8.7.1

APM Agent language and version: react.js / node.js

Browser version:

Original install method (e.g. download page, yum, deb, from source, etc.) and version:

Fresh install or upgraded from other version?

Is there anything special in your setup? For example, are you using the Logstash or Kafka outputs? Are you using a load balancer in front of the APM Servers? Have you changed index pattern, generated custom templates, changed agent configuration etc.

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):

I keep seeing the following error message in my ECK cluster:

[buyerportal-0 buyerportal] {"log.level":"error","@timestamp":"2023-05-19T06:46:40.752Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"APM Server transport error (503): Unexpected APM Server response when polling config\nAgent configuration infrastructure is not ready. Please retry later."}

What's the root cause? How to resolve it?

Steps to reproduce:
1.
2.
3.

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

2

Hi @Kok_How_Teh,

It looks to me like you've quoted two separate log errors from the Node.js APM agent:

  1. APM server did not respond within 10s of gzip stream finish
  2. APM Server transport error (503): Unexpected APM Server response when polling config\nAgent configuration infrastructure is not ready. Please retry later.

Both of these indicate an issue with the APM server that the APM agent is configured to talk to. The first error message is from when the APM agent is attempting to send collected APM data to the APM server. The second error message is when the APM agent periodically polls the APM server for custom configuration.

Has the issue since cleared up?

Troubleshoot ECK | Elastic Cloud on Kubernetes [2.7] | Elastic includes some troubleshooting notes specific to ECK.

Are you able to look for APM server logs to see if the APM server is healthy?

3

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.