Elastic APM not working due to error "Exception Type: TransportException Exception Value: HTTP 503: {"accepted":0,"errors":[{"message":"queue is full"}]}"

[SUNA (Sudhakar Amineni)]

Hi Team,

Elastic APM metrics not showing due to error "Exception Type: TransportException
Exception Value: HTTP 503: {"accepted":0,"errors":[{"message":"queue is full"}]}"

Could you please help to clear queue to fix APM issue.

Currently my APM dashboard not working due to error below error

file_path: /var/www/api.wakefit.co/venv/lib/python3.6/site-packages/elasticapm/transport/base.py
func_name: handle_transport_fail
line_no: 334
Failed to submit message: 'HTTP 503: {"accepted":0,"errors":[{"message":"queue is full"}]}\n'
TransportException


Server time: Tue, 27 Dec 2022 06:36:38 +0530

Traceback:

File "/var/www/api.wakefit.co/venv/lib/python3.6/site-packages/elasticapm/transport/base.py" in _flush
  275.                 self.send(data, forced_flush=forced_flush)

File "/var/www/api.wakefit.co/venv/lib/python3.6/site-packages/elasticapm/transport/http.py" in send
  108.                 raise TransportException(message, data, print_trace=print_trace)

Exception Type: TransportException
Exception Value: HTTP 503: {"accepted":0,"errors":[{"message":"queue is full"}]}

Currently my APM dashboard error as below

![image|690x232](upload://7rH2lrrze1n5QV90soZdu4z2AtA.jpeg)

Could you please help to debug this issue and how to clear queue

[SUNA (Sudhakar Amineni)]

Below APM Dashboard error

[simitt (Silvia Mitter)]

Hi @SUNA ,

When the internal queue fills up, this is mostly due to mapping mismatches if the index template setup wasn't successful or due to the APM Server being overloaded. Since you don't see any data in the Kibana UI, I suspect that something with the setup might be wrong. Currently we have too little details to make any more specific suggestions. As a start, you should take a look at the APM Server logs and check if there are any errors, indicating what is the root cause of the queue filling up.

If you share any error logs in this forum, please redact any sensitive information before sharing, and also include some meta information around the version, installation method and how you set up the Elasticsearch index templates.

The APM Server guide has a troubleshooting section for when the queue fills up, please give it a read.

[SUNA (Sudhakar Amineni)]

Thanks @simitt for your reply. will troubleshoot as you suggested.