# APM adding query params causing 400 bad request error during ES query

**URL:** <https://discuss.elastic.co/t/apm-adding-query-params-causing-400-bad-request-error-during-es-query/186749>\
**Category:** APM\
**Tags:** python\
**Created:** [June 20, 2019, 7:32pm UTC](https://discuss.elastic.co/t/apm-adding-query-params-causing-400-bad-request-error-during-es-query/186749 "2019-06-20T19:32:46Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![gram](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gram/32/48473_2.png) [@gram](https://discuss.elastic.co/u/gram)\
**Post date:** [June 20, 2019, 7:32pm UTC](https://discuss.elastic.co/t/apm-adding-query-params-causing-400-bad-request-error-during-es-query/186749/1 "2019-06-20T19:32:46Z")

</div>

**Elasticsearch version** : 7.0.1

**APM Agent language and version** : Python 4.2.2

**Description of the problem including expected versus actual behavior. Please include screenshots (if relevant)**:

I am seeing the follow error multiple times in my logs:

```auto
RequestError(400, 'illegal_argument_exception', 
 'request [/my_index/_search] contains unrecognized parameters: [__elastic_apm_api_method_name], [__elastic_apm_body_ref]')  

```

It appears that APM is adding query params to the ES search request. When I run something similar with Kibana I see the same error:  
`GET /my_index/search?elastic_apm`

```auto
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "request [/my_index/_search] contains unrecognized parameter: [elastic_apm]"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "request [/my_index/_search] contains unrecognized parameter: [elastic_apm]"
  },
  "status": 400
}

```

Can I force APM to not do this? Or, force ES to not throw because of unused params? Thanks!

**Steps to reproduce** :

1. Configure APM Python Agent
2. Have endpoint with ES query on that service

- service is using gunicorn & gevent & Flask

1. Run many calls against that endpoint and some will throw the exception

**Provide logs and/or server output (if relevant)**:

---

<div class="post-metadata">

**Author:** ![beniwohli](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/beniwohli/32/21256_2.png) [@beniwohli](https://discuss.elastic.co/u/beniwohli)\
**Post date:** [June 20, 2019, 7:56pm UTC](https://discuss.elastic.co/t/apm-adding-query-params-causing-400-bad-request-error-during-es-query/186749/2 "2019-06-20T19:56:32Z")

</div>

Hi Graham

thanks for the heads up! This does indeed look like a bug.

There were a few changes in the elasticsearch client for 7.0 that affect the APM agent. It's possible that this is the cause of the bug. I recently merged a fix, but didn't get around to publish a release yet.

If you have a test/staging environment where you could safely test this, you could try and install the agent from master to see if it fixes the issue:

```
pip install "https://github.com/elastic/apm-agent-python/archive/master.zip#egg=elastic-apm==100"

```

If that is not an option, you can disable the elasticsearch client instrumentation by setting the two environment variables `SKIP_INSTRUMENT_ELASTICSEARCH` and `SKIP_INSTRUMENT_ELASTICSEARCH_CONNECTION` to any value for your gunicorn process.

Cheers  
Beni

---

<div class="post-metadata">

**Author:** ![gram](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gram/32/48473_2.png) [@gram](https://discuss.elastic.co/u/gram)\
**Post date:** [June 20, 2019, 8:14pm UTC](https://discuss.elastic.co/t/apm-adding-query-params-causing-400-bad-request-error-during-es-query/186749/3 "2019-06-20T20:14:11Z")

</div>

Thanks! I'll disable it until I can test the update.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 11, 2019, 4:14pm UTC](https://discuss.elastic.co/t/apm-adding-query-params-causing-400-bad-request-error-during-es-query/186749/4 "2019-07-11T16:14:15Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.