# Failed to submit message (aws lambda using python)

**URL:** <https://discuss.elastic.co/t/failed-to-submit-message-aws-lambda-using-python/332576>\
**Category:** APM\
**Tags:** python\
**Created:** [May 4, 2023, 3:18pm UTC](https://discuss.elastic.co/t/failed-to-submit-message-aws-lambda-using-python/332576 "2023-05-04T15:18:29Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![gabriel-f-santos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gabriel-f-santos/32/120655_2.png) [@gabriel-f-santos](https://discuss.elastic.co/u/gabriel-f-santos)\
**Post date:** [May 4, 2023, 3:18pm UTC](https://discuss.elastic.co/t/failed-to-submit-message-aws-lambda-using-python/332576/1 "2023-05-04T15:18:29Z")

</div>

Hello guys,

I am using elastic in lambda, with:

- elastic-apm==6.14
- layer:elastic-apm-extension-ver-1-3-0-x86\_64:1
- python 3.9 in aws lambda

I am receiving an error in some operations, most of the executions works fine, but i have some erros like this:

Failed to submit message: "Unable to reach APM Server: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

Did someone resolved this error ?  
Thanks guys

---

<div class="post-metadata">

**Author:** ![basepi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/basepi/32/53056_2.png) [@basepi](https://discuss.elastic.co/u/basepi)\
**Post date:** [May 5, 2023, 5:05pm UTC](https://discuss.elastic.co/t/failed-to-submit-message-aws-lambda-using-python/332576/2 "2023-05-05T17:05:17Z")

</div>

Can you show your (redacted) environment variables?

@lahsivjar have you seen anything like this from the extension?

---

<div class="post-metadata">

**Author:** ![gabriel-f-santos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gabriel-f-santos/32/120655_2.png) [@gabriel-f-santos](https://discuss.elastic.co/u/gabriel-f-santos)\
**Post date:** [May 5, 2023, 5:24pm UTC](https://discuss.elastic.co/t/failed-to-submit-message-aws-lambda-using-python/332576/3 "2023-05-05T17:24:07Z")

</div>

Sure. I am using SAM, the template.yml contains the environment variables:

```
    ELASTIC_APM_SERVICE_NAME: "service-name"
    ELASTIC_APM_SEND_STRATEGY: background
    ELASTIC_APM_SECRET_TOKEN:
      Ref: ElasticAPMSecretToken
    ELASTIC_APM_LAMBDA_APM_SERVER: http://elastic.prd.mybusiness.net:8200
    ELASTIC_APM_ENVIRONMENT: PRD

```

Layer:  
Layers:  
- arn:aws:lambda:XXXXXXX:layer:elastic-apm-extension-ver-1-3-0-x86\_64:1

Lambda execution works fine, all errors are handled by elastic, but in this cases we have the error traceback logging in another tool that we use for logs.

Complete traceback:

[ERROR] (redacted) ailed to submit message: "Unable to reach APM Server: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) (url: [http://localhost:8200/intake/v2/events](http://localhost:8200/intake/v2/events))"  
Traceback (most recent call last):  
File "/opt/python/urllib3/connectionpool.py", line 703, in urlopen  
httplib\_response = self.\_make\_request(  
File "/opt/python/urllib3/connectionpool.py", line 449, in \_make\_request  
six.raise\_from(e, None)  
File "", line 3, in raise\_from  
File "/opt/python/urllib3/connectionpool.py", line 444, in \_make\_request  
httplib\_response = conn.getresponse()  
File "/opt/python/sentry\_sdk/integrations/stdlib.py", line 121, in getresponse  
rv = real\_getresponse(self, \*args, \*\*kwargs)  
File "/var/lang/lib/python3.8/http/client.py", line 1348, in getresponse  
response.begin()  
File "/var/lang/lib/python3.8/http/client.py", line 316, in begin  
version, status, reason = self.\_read\_status()  
File "/var/lang/lib/python3.8/http/client.py", line 285, in \_read\_status  
raise RemoteDisconnected("Remote end closed connection without"  
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):  
File "/opt/python/elasticapm/transport/http.py", line 90, in send  
response = self.http.urlopen(  
File "/opt/python/urllib3/poolmanager.py", line 376, in urlopen  
response = conn.urlopen(method, u.request\_uri, \*\*kw)  
File "/opt/python/elasticapm/instrumentation/packages/base.py", line 206, in call\_if\_sampling  
return wrapped(\*args, \*\*kwargs)  
File "/opt/python/urllib3/connectionpool.py", line 787, in urlopen  
retries = retries.increment(  
File "/opt/python/urllib3/util/retry.py", line 550, in increment  
raise six.reraise(type(error), error, \_stacktrace)  
File "/opt/python/urllib3/packages/six.py", line 769, in reraise  
raise value.with\_traceback(tb)  
File "/opt/python/urllib3/connectionpool.py", line 703, in urlopen  
httplib\_response = self.\_make\_request(  
File "/opt/python/urllib3/connectionpool.py", line 449, in \_make\_request  
six.raise\_from(e, None)  
File "", line 3, in raise\_from  
File "/opt/python/urllib3/connectionpool.py", line 444, in \_make\_request  
httplib\_response = conn.getresponse()  
File "/opt/python/sentry\_sdk/integrations/stdlib.py", line 121, in getresponse  
rv = real\_getresponse(self, \*args, \*\*kwargs)  
File "/var/lang/lib/python3.8/http/client.py", line 1348, in getresponse  
response.begin()  
File "/var/lang/lib/python3.8/http/client.py", line 316, in begin  
version, status, reason = self.\_read\_status()  
File "/var/lang/lib/python3.8/http/client.py", line 285, in \_read\_status  
raise RemoteDisconnected("Remote end closed connection without"  
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):  
File "/opt/python/elasticapm/transport/base.py", line 279, in \_flush  
self.send(data, forced\_flush=forced\_flush)  
File "/opt/python/elasticapm/transport/http.py", line 104, in send  
raise TransportException(message, data, print\_trace=print\_trace)  
elasticapm.transport.exceptions.TransportException: Unable to reach APM Server: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) (url: [http://localhost:8200/intake/v2/events](http://localhost:8200/intake/v2/events))  
[ERROR] 2023-05-03T19:45:26.858Z 4cd917fd-3fb3-5206-97c1-4b31843f4bce Failed to submit message: "Unable to reach APM Server: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) (url: [http://localhost:8200/intake/v2/events](http://localhost:8200/intake/v2/events))" Traceback (most recent call last): File "/opt/python/urllib3/connectionpool.py", line 703, in urlopen httplib\_response = self.\_make\_request( File "/opt/python/urllib3/connectionpool.py", line 449, in \_make\_request six.raise\_from(e, None) File "", line 3, in raise\_from File "/opt/python/urllib3/connectionpool.py", line 444, in \_make\_request httplib\_response = conn.getresponse() File "/opt/python/sentry\_sdk/integrations/stdlib.py", line 121, in getresponse rv = real\_getresponse(self, \*args, \*\*kwargs) File "/var/lang/lib/python3.8/http/client.py", line 1348, in getresponse response.begin() File "/var/lang/lib/python3.8/http/client.py", line 316, in begin version, status, reason = self.\_read\_status() File "/var/lang/lib/python3.8/http/client.py", line 285, in \_read\_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/python/elasticapm/transport/http.py", line 90, in send response = self.http.urlopen( File "/opt/python/urllib3/poolmanager.py", line 376, in urlopen response = conn.urlopen(method, u.request\_uri, \*\*kw) File "/opt/python/elasticapm/instrumentation/packages/base.py", line 206, in call\_if\_sampling return wrapped(\*args, \*\*kwargs) File "/opt/python/urllib3/connectionpool.py", line 787, in urlopen retries = retries.increment( File "/opt/python/urllib3/util/retry.py", line 550, in increment raise six.reraise(type(error), error, \_stacktrace) File "/opt/python/urllib3/packages/six.py", line 769, in reraise raise value.with\_traceback(tb) File "/opt/python/urllib3/connectionpool.py", line 703, in urlopen httplib\_response = self.\_make\_request( File "/opt/python/urllib3/connectionpool.py", line 449, in \_make\_request six.raise\_from(e, None) File "", line 3, in raise\_from File "/opt/python/urllib3/connectionpool.py", line 444, in \_make\_request httplib\_response = conn.getresponse() File "/opt/python/sentry\_sdk/integrations/stdlib.py", line 121, in getresponse rv = real\_getresponse(self, \*args, \*\*kwargs) File "/var/lang/lib/python3.8/http/client.py", line 1348, in getresponse response.begin() File "/var/lang/lib/python3.8/http/client.py", line 316, in begin version, status, reason = self.\_read\_status() File "/var/lang/lib/python3.8/http/client.py", line 285, in \_read\_status raise RemoteDisconnected("Remote end closed connection without" urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/opt/python/elasticapm/transport/base.py", line 279, in \_flush self.send(data, forced\_flush=forced\_flush) File "/opt/python/elasticapm/transport/http.py", line 104, in send raise TransportException(message, data, print\_trace=print\_trace) elasticapm.transport.exceptions.TransportException: Unable to reach APM Server: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) (url: [http://localhost:8200/intake/v2/events](http://localhost:8200/intake/v2/events))

Thanks @basepi

---

<div class="post-metadata">

**Author:** ![lahsivjar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lahsivjar/32/101451_2.png) [@lahsivjar](https://discuss.elastic.co/u/lahsivjar)\
**Post date:** [May 8, 2023, 4:38am UTC](https://discuss.elastic.co/t/failed-to-submit-message-aws-lambda-using-python/332576/4 "2023-05-08T04:38:23Z")

</div>

> [@basepi](#):
>
> @lahsivjar have you seen anything like this from the extension?

No, haven't seen such errors before, the extension seems to be aborting the connection from the python agent.

@gabriel-f-santos Will you be able to share the extension logs in debug mode when the issue happens? Debug mode can be enabled by setting the environment variable `ELASTIC_APM_LOG_LEVEL` to `debug`.

You can get the extension logs from Cloudwatch.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 5, 2023, 4:39am UTC](https://discuss.elastic.co/t/failed-to-submit-message-aws-lambda-using-python/332576/5 "2023-06-05T04:39:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
