# Python RQ

**URL:** <https://discuss.elastic.co/t/python-rq/206045>\
**Category:** APM\
**Tags:** python\
**Created:** [October 31, 2019, 1:33pm UTC](https://discuss.elastic.co/t/python-rq/206045 "2019-10-31T13:33:46Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Josef\_Rousek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/josef_rousek/32/56948_2.png) [@Josef\_Rousek](https://discuss.elastic.co/u/Josef_Rousek)\
**Post date:** [October 31, 2019, 1:33pm UTC](https://discuss.elastic.co/t/python-rq/206045/1 "2019-10-31T13:33:46Z")

</div>

Hi,  
I'm having hard time setting up APM with Python RQ, which is using fork. I was following [Python agent only recording some background tasks](https://discuss.elastic.co/t/python-agent-only-recording-some-background-tasks/149707), which led me to creating custom `rq.job`. However with removal of `MAX_QUEUE_SIZE` I don't know how to convince agent to send the transaction synchronously in the same thread as the background job.

The only way I got it to send a transaction was to call `client._transport._process_queue()` but then it never gets out of the loop.

```auto
import elasticapm
from elasticapm.contrib.django.client import get_client
from rq.job import Job

class TracingJob(Job):
    def perform(self):
        client = get_client()
        client.begin_transaction("backgroundjob")
        context = {"args": self.args, "kwargs": self.kwargs}
        print(context)
        print(self.func_name)
        elasticapm.set_custom_context(context)
        try:
            super().perform()
            client.end_transaction(self.func_name, "SUCCESS")
        except:
            client.capture_exception()
            raise
        finally:
            client._transport._process_queue()

```

I'm using elastic apm python agent 5.1.2.

---

<div class="post-metadata">

**Author:** ![basepi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/basepi/32/53056_2.png) [@basepi](https://discuss.elastic.co/u/basepi)\
**Post date:** [November 11, 2019, 10:59pm UTC](https://discuss.elastic.co/t/python-rq/206045/2 "2019-11-11T22:59:15Z")

</div>

Great question! It looks like `MAX_QUEUE_SIZE` was replaced with [`API_REQUEST_SIZE`](https://www.elastic.co/guide/en/apm/agent/python/current/configuration.html#config-api-request-size). So, if you want to make the agent send after each transaction, you can set `API_REQUEST_SIZE` to `"1b"` (1 byte) and it should give you the same behavior as with the old `MAX_QUEUE_SIZE=1`.

---

<div class="post-metadata">

**Author:** ![Josef\_Rousek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/josef_rousek/32/56948_2.png) [@Josef\_Rousek](https://discuss.elastic.co/u/Josef_Rousek)\
**Post date:** [November 14, 2019, 8:48am UTC](https://discuss.elastic.co/t/python-rq/206045/3 "2019-11-14T08:48:10Z")

</div>

Hi Colton,

Thanks for the suggestion. However it didn't do the trick. I tried to debug where the things go wrong but the code is just advanced for me. I created minimal Django app with Elastic APM working on the view and my attempt to implement `TracingJob`.

> **[stlk/elastic-rq-demo](https://github.com/stlk/elastic-rq-demo)**
>
> Contribute to stlk/elastic-rq-demo development by creating an account on GitHub.

Readme describes how to setup the app and how to replicate my issue. It would be great if someone with understanding of the queue and the transport layer could take a look at it.

---

<div class="post-metadata">

**Author:** ![basepi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/basepi/32/53056_2.png) [@basepi](https://discuss.elastic.co/u/basepi)\
**Post date:** [November 14, 2019, 5:02pm UTC](https://discuss.elastic.co/t/python-rq/206045/4 "2019-11-14T17:02:26Z")

</div>

Thanks for providing a way to reproduce this! I don't want to lose track of this so can you replicate the important bits of this thread in [an issue](https://github.com/elastic/apm-agent-python/issues) (including your demo repo) so that we can track it? I definitely want to take a look when I find a moment.

(We'll also want to track down if there is a bug in that `API_REQUEST_SIZE` config or if we're just never getting there for some reason.)

---

<div class="post-metadata">

**Author:** ![Josef\_Rousek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/josef_rousek/32/56948_2.png) [@Josef\_Rousek](https://discuss.elastic.co/u/Josef_Rousek)\
**Post date:** [November 14, 2019, 6:18pm UTC](https://discuss.elastic.co/t/python-rq/206045/5 "2019-11-14T18:18:54Z")

</div>

Sure. Here's the issue with rephrased problem [https://github.com/elastic/apm-agent-python/issues/637](https://github.com/elastic/apm-agent-python/issues/637)

Thanks a lot!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 5, 2019, 2:18pm UTC](https://discuss.elastic.co/t/python-rq/206045/6 "2019-12-05T14:18:57Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.