APM python agent stoped showing boto3 spans


(Ahmad Al Masry) #1

Hi;
We are using elastic cloud and we were using 6.4.3 and apm agent 3.0.3, then, the boto3 instrumentation went so well, but after we upgraded to 6.5.0 and apm agent 4.0.1 the distributed tracing went well but the boto3 instrumentation stopped working.

I do not know if they are related, but if we disabled the distributed tracing, will boto3 instrumentation return to work? If yes, how can we disable the distributed tracing.

Note: boto3 calls appear as GET with now argument of to 169.* ip.

Thank you for your cooperation.


(Benjamin Wohlwend @ APM) #2

Hi @Ahmad_Al-Masry

Sorry about that! Distributed tracing definitely shouldn't break boto3 support. I'll investigate this and report back when I find the problem. To help me get started, can you tell me which version of boto3 and botocore you use?


(Ahmad Al Masry) #3

Hi;

We use the following both 1.4.1 and 1.4.4, both were working fine before the update.

And we use the following configs:

ELASTIC_APM = {

'SERVICE_NAME': 'lpm-aggregator',

'ENVIRONMENT': 'prod',

'CAPTURE_BODY': 'error',

'DEBUG': False,

'INSTRUMENT': True,

}

INSTALLED_APPS = [

.

.

.

'elasticapm.contrib.django',

.

.

.

]

MIDDLEWARE_CLASSES = [

'elasticapm.contrib.django.middleware.TracingMiddleware',

.

.

.

]

These services are using uwsgi to run.

Thank you for your cooperation.

BR


(Ahmad Al Masry) #4

Hi;
Is there any misconfiguration from our side, or there is another issue?
BR


(Benjamin Wohlwend @ APM) #5

I'm still investigating the issue. I added a test to our test suite that tests the full HTTP flow (https://github.com/elastic/apm-agent-python/pull/341), and that seems to work, so there must be something else going on.

One thing that I would suggest is to upgrade boto3 and botocore to a more recent version if possible. Versions of boto3 before 1.8.0 vendored the requests module, which in turn vendored the urllib3 module, which makes things very complicated for us.


(Ahmad Al Masry) #6

Hi;
We upgraded to 1.9.50 "Latest boto3" and the same problem.
I want to note that it was running perfectly when we used apm agent 3.0.3 and elastic 6.4.3.
Thank you for your help.


(Jung Juhong) #7

I got same problem.
I'm using boto3 1.9.50, apm python agent 3.0.4, and apm server 6.5.1.