# Python agent only recording some background tasks

**URL:** <https://discuss.elastic.co/t/python-agent-only-recording-some-background-tasks/149707>\
**Category:** APM\
**Created:** [September 24, 2018, 4:03pm UTC](https://discuss.elastic.co/t/python-agent-only-recording-some-background-tasks/149707 "2018-09-24T16:03:23Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![sburns](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sburns/32/35821_2.png) [@sburns](https://discuss.elastic.co/u/sburns)\
**Post date:** [September 24, 2018, 4:03pm UTC](https://discuss.elastic.co/t/python-agent-only-recording-some-background-tasks/149707/1 "2018-09-24T16:03:23Z")

</div>

Hi there! 👋

We're using this decorator to record background job (using RQ) performance.

```
# jobs.py

ELASTIC_APM = {
    'INCLUDE_PATH': ['health'],
    'SERVICE_NAME': 'health-{}'.format(ENVIRONMENT),
    'SERVER_URL': os.environ['APM_HOST'],
}

def apm_decorator(*apm_args):
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            client = elasticapm.Client(settings.ELASTIC_APM)
            client.begin_transaction(*apm_args)
            context = {
                'args': args,
                'kwargs': kwargs,
            }
            elasticapm.set_custom_context(context)
            result = func(*args, **kwargs)
            full_name = '{}.{}'.format(func. __module__ , func. __name__ )
            client.end_transaction(full_name, 'SUCCESS')
            return result
        return wrapper
    return decorator

@apm_decorator('jobs')
def real_function():
    pass

@job('queue')
@apm_decorator('jobs')
def background_task():
    real_function()
    # other stuff

# views.py
def view(request):
    background_task.delay()

```

(The `'jobs'` arg passed to the `apm_decorator` puts these transactions into the "jobs" tab in APM.)

`real_function` is reported but nothing about `background_task` appears in APM so anything before or after `real_function` is not reported. (HTTP requests from Django are reported normally, so our APM server seems ok.)

Python 3.6.4 (also occurred in python 2.7.X), APM Agent version 3.0.0

Thanks for your help!

---

<div class="post-metadata">

**Author:** ![axw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/axw/32/28197_2.png) [@axw](https://discuss.elastic.co/u/axw)\
**Post date:** [September 25, 2018, 2:02am UTC](https://discuss.elastic.co/t/python-agent-only-recording-some-background-tasks/149707/2 "2018-09-25T02:02:02Z")

</div>

Hi Scott, thanks for trying out Elastic APM!

RQ uses a fork/exec pattern, so I suspect what's happening is that some of your jobs are completing before the transactions can be sent off to the APM Server.

By default the agent won't send data to the server until 10 seconds have elapsed, or 500 transactions have been recorded. These can be controlled with [ELASTIC\_APM\_FLUSH\_INTERVAL](https://www.elastic.co/guide/en/apm/agent/python/current/configuration.html#config-flush-interval) and [ELASTIC\_APM\_MAX\_QUEUE\_SIZE](https://www.elastic.co/guide/en/apm/agent/python/current/configuration.html#config-max-queue-size) respectively.

Adding `MAX_QUEUE_SIZE: 1` to the `ELASTIC_APM` settings dict will force each job transaction to be sent immediately.

---

<div class="post-metadata">

**Author:** ![sburns](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sburns/32/35821_2.png) [@sburns](https://discuss.elastic.co/u/sburns)\
**Post date:** [September 26, 2018, 5:21pm UTC](https://discuss.elastic.co/t/python-agent-only-recording-some-background-tasks/149707/3 "2018-09-26T17:21:31Z")

</div>

Thanks for your reply Andrew, this was very helpful. For those that end up on this page, this is what I ended up doing:

- Set the ELASTIC\_APM\_MAX\_QUEUE\_SIZE envvar
- Subclass rq.job.Job and override the `perform` function with basically what I used above. Begin the transaction, set custom context w/ the jobs args and kwargs, `super().perform()`, and then end the transaction and close the client. django-rq allows for setting the global custom Job class ([https://github.com/rq/django-rq#custom-job-and-worker-classes](https://github.com/rq/django-rq#custom-job-and-worker-classes)) so it was relatively easy to hook that up.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 17, 2018, 1:21pm UTC](https://discuss.elastic.co/t/python-agent-only-recording-some-background-tasks/149707/4 "2018-10-17T13:21:35Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.