APM python agent stoped showing boto3 spans

Elastic Observability APM
1

Hi;
We are using elastic cloud and we were using 6.4.3 and apm agent 3.0.3, then, the boto3 instrumentation went so well, but after we upgraded to 6.5.0 and apm agent 4.0.1 the distributed tracing went well but the boto3 instrumentation stopped working.

I do not know if they are related, but if we disabled the distributed tracing, will boto3 instrumentation return to work? If yes, how can we disable the distributed tracing.

Note: boto3 calls appear as GET with now argument of to 169.* ip.

Thank you for your cooperation.

33%20PM
Screen Shot 2018-11-20 at 1.48.33 PM.png3312×680 121 KB
55%20PM
Screen Shot 2018-11-20 at 1.48.55 PM.png3264×1850 326 KB

2

Hi @Ahmad_Al-Masry

Sorry about that! Distributed tracing definitely shouldn't break boto3 support. I'll investigate this and report back when I find the problem. To help me get started, can you tell me which version of boto3 and botocore you use?

3

Hi;

We use the following both 1.4.1 and 1.4.4, both were working fine before the update.

And we use the following configs:

ELASTIC_APM = {

'SERVICE_NAME': 'lpm-aggregator',

'ENVIRONMENT': 'prod',

'CAPTURE_BODY': 'error',

'DEBUG': False,

'INSTRUMENT': True,

}

INSTALLED_APPS = [

.

.

.

'elasticapm.contrib.django',

.

.

.

]

MIDDLEWARE_CLASSES = [

'elasticapm.contrib.django.middleware.TracingMiddleware',

.

.

.

]

These services are using uwsgi to run.

Thank you for your cooperation.

BR

4

Hi;
Is there any misconfiguration from our side, or there is another issue?
BR

5

I'm still investigating the issue. I added a test to our test suite that tests the full HTTP flow (https://github.com/elastic/apm-agent-python/pull/341), and that seems to work, so there must be something else going on.

One thing that I would suggest is to upgrade boto3 and botocore to a more recent version if possible. Versions of boto3 before 1.8.0 vendored the requests module, which in turn vendored the urllib3 module, which makes things very complicated for us.

6

Hi;
We upgraded to 1.9.50 "Latest boto3" and the same problem.
I want to note that it was running perfectly when we used apm agent 3.0.3 and elastic 6.4.3.
Thank you for your help.

7

I got same problem.
I'm using boto3 1.9.50, apm python agent 3.0.4, and apm server 6.5.1.

8

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.

9

Hey everybody

turns out this was a bug in the agent :frowning: sorry for not recognizing that earlier. It should be fixed in both 3.0.5 and 4.0.3 of the agent.