# APM viewer in kibana broken when transaction name has a slash in it

**URL:** <https://discuss.elastic.co/t/apm-viewer-in-kibana-broken-when-transaction-name-has-a-slash-in-it/176044>\
**Category:** APM\
**Created:** [April 9, 2019, 2:21pm UTC](https://discuss.elastic.co/t/apm-viewer-in-kibana-broken-when-transaction-name-has-a-slash-in-it/176044 "2019-04-09T14:21:13Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![sgaron\_cse](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sgaron_cse/32/43777_2.png) [@sgaron\_cse](https://discuss.elastic.co/u/sgaron_cse)\
**Post date:** [April 9, 2019, 2:21pm UTC](https://discuss.elastic.co/t/apm-viewer-in-kibana-broken-when-transaction-name-has-a-slash-in-it/176044/1 "2019-04-09T14:21:13Z")

</div>

**Kibana version** : 6.7.1

**Elasticsearch version** : 6.7.1

**APM Server version** : 6.7.1

**APM Agent language and version** : Python 4.2.1 [Flask integration]

**Browser version** : Chrome 73.0.3683.103

**Original install method (e.g. download page, yum, deb, from source, etc.) and version**: Official docker containers

**Fresh install or upgraded from other version?** Fresh install

**Is there anything special in your setup?** No special config

**Description of the problem including expected versus actual behavior. Please include screenshots (if relevant)**:

When using the flask integration of ElasticAPM, metrics are collected correctly into the elasticsearch cluster but when trying to view the data, there is a bug in how kibana calls its own APM APIs because of the slashes the ElasticAPM put in the transaction name.

For example, viewing the transactions for "GET /index.html" makes the following two API calls return a 404 not found:

- [http://localhost:5601/api/apm/services/my\_app/transaction\_groups/request/GET%20%2Findex.html/charts?start=2019-04-09T12%3A45%3A10.735Z&end=2019-04-09T13%3A45%3A10.736Z](http://localhost:5601/api/apm/services/my_app/transaction_groups/request/GET%20%2Findex.html/charts?start=2019-04-09T12%3A45%3A10.735Z&end=2019-04-09T13%3A45%3A10.736Z)
- [http://localhost:5601/api/apm/services/my\_app/transaction\_groups/request/GET%20%2Findex.html/distribution?start=2019-04-09T12%3A45%3A10.735Z&end=2019-04-09T13%3A45%3A10.736Z](http://localhost:5601/api/apm/services/my_app/transaction_groups/request/GET%20%2Findex.html/distribution?start=2019-04-09T12%3A45%3A10.735Z&end=2019-04-09T13%3A45%3A10.736Z)

It looks like the **~20** and the **~2F** for the original kibana url ([http://localhost:5601/app/apm#/my\_app/transactions/request/GET~20~2Findex.html](http://localhost:5601/app/apm#/my_app/transactions/request/GET~20~2Findex.html)) are translated to **%20** and **%2F** which confuses the API endpoint. If you replace the **%** in the failing urls with **~** they return the right results.

**Steps to reproduce** :

1. Gather ElasticAPM metrics from any flask app
2. Click on the APM metrics in kibana for a given page
3. No transaction will be found and you will get a 404 for the chart and distribution APM APIs

**Errors in browser console (if relevant)**:

```
GET http://localhost:5601/api/apm/services/my_app/transaction_groups/request/GET%20%2Findex.html/distribution?start=2019-04-09T12%3A45%3A10.735Z&end=2019-04-09T13%3A45%3A10.736Z 404 (Not Found)
(anonymous) @ commons.bundle.js:3
Promise.then (async)
(anonymous) @ commons.bundle.js:3
step @ vendors.bundle.dll.js:531
(anonymous) @ vendors.bundle.dll.js:531
(anonymous) @ vendors.bundle.dll.js:531
__awaiter @ vendors.bundle.dll.js:531
kfetch @ commons.bundle.js:3
_callee$ @ apm.bundle.js:2
tryCatch @ vendors.bundle.dll.js:497
invoke @ vendors.bundle.dll.js:497
prototype.(anonymous function) @ vendors.bundle.dll.js:497
step @ apm.bundle.js:2
(anonymous) @ apm.bundle.js:2
(anonymous) @ apm.bundle.js:2
callApi @ apm.bundle.js:2
_callee2$ @ apm.bundle.js:2
tryCatch @ vendors.bundle.dll.js:497
invoke @ vendors.bundle.dll.js:497
prototype.(anonymous function) @ vendors.bundle.dll.js:497
step @ apm.bundle.js:2
(anonymous) @ apm.bundle.js:2
Promise.then (async)
step @ apm.bundle.js:2
(anonymous) @ apm.bundle.js:2
(anonymous) @ apm.bundle.js:2
loadTransactionDistribution @ apm.bundle.js:2
_callee$ @ vendors.bundle.dll.js:491
tryCatch @ vendors.bundle.dll.js:497
invoke @ vendors.bundle.dll.js:497
prototype.(anonymous function) @ vendors.bundle.dll.js:497
step @ vendors.bundle.dll.js:491
(anonymous) @ vendors.bundle.dll.js:491
(anonymous) @ vendors.bundle.dll.js:491
maybeFetchData @ vendors.bundle.dll.js:491
_callee3$ @ vendors.bundle.dll.js:491
tryCatch @ vendors.bundle.dll.js:497
invoke @ vendors.bundle.dll.js:497
prototype.(anonymous function) @ vendors.bundle.dll.js:497
step @ vendors.bundle.dll.js:491
(anonymous) @ vendors.bundle.dll.js:491
(anonymous) @ vendors.bundle.dll.js:491
componentDidUpdate @ vendors.bundle.dll.js:491
Hi @ vendors.bundle.dll.js:212
Fi @ vendors.bundle.dll.js:212
Gi @ vendors.bundle.dll.js:212
ii @ vendors.bundle.dll.js:212
xg @ vendors.bundle.dll.js:212
enqueueSetState @ vendors.bundle.dll.js:212
G.setState @ vendors.bundle.dll.js:180
onStateChange @ vendors.bundle.dll.js:149
dispatch @ vendors.bundle.dll.js:76
(anonymous) @ vendors.bundle.dll.js:497
(anonymous) @ apm.bundle.js:2
setInterval (async)
updateRefreshRate @ apm.bundle.js:2
(anonymous) @ apm.bundle.js:2
listener @ vendors.bundle.dll.js:62
(anonymous) @ vendors.bundle.dll.js:62
notifyListeners @ vendors.bundle.dll.js:62
setState @ vendors.bundle.dll.js:111
(anonymous) @ vendors.bundle.dll.js:111
confirmTransitionTo @ vendors.bundle.dll.js:62
handlePop @ vendors.bundle.dll.js:111
handleHashChange @ vendors.bundle.dll.js:111

```

**Provide logs and/or server output (if relevant)**:  
No error server side...

**Workaround** :  
For now, if I want to see all the the transactions properly I have to monkey patch the build\_name\_with\_http\_method\_prefix from the ElasticAPM utils library to strip all slashes.

```
def monkey_build_name_with_http_method_prefix(name, request):
    name = name.strip("/").replace("/", ".")
    return " ".join((request.method, name)) if name else "none"
   
from elasticapm import utils
utils.build_name_with_http_method_prefix = monkey_build_name_with_http_method_prefix

```

This is obviously not the proper way to fix this. Kibana APM APIs should be able to handle request with slashes in the name.

---

<div class="post-metadata">

**Author:** ![sqren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sqren/32/26110_2.png) [@sqren](https://discuss.elastic.co/u/sqren)\
**Post date:** [April 9, 2019, 10:54pm UTC](https://discuss.elastic.co/t/apm-viewer-in-kibana-broken-when-transaction-name-has-a-slash-in-it/176044/2 "2019-04-09T22:54:46Z")

</div>

Hi there,

Thanks for reaching out. I agree, forward slashes in transaction names should definitely be supported. Unfortunately I have not been able reproduce you issue, so I will need a little more info from you.

From the URLs you have posted, everything looks right. Kibana uses a custom encoding scheme which uses `~` (tilde) instead of `%` (percent), and the API requests to the backend uses normal encoding.

Depending on context you should therefore see three different representations of the transaction name:  
**decoded** : GET /index.html  
**normal encoding** : GET%20%2Findex.html  
**custom encoding** : GET~20~2Findex.html

I'm therefore quite curious as to why the API requests return 404. I would expect the following curl request to return 200:

```auto
curl -v http://localhost:5601/api/apm/services/my_app/transaction_groups/request/GET%20%2Findex.html/distribution?start=2019-04-09T12%3A45%3A10.735Z&end=2019-04-09T13%3A45%3A10.736Z

```

I don't suppose you have any middleware or proxy when you call `http://localhost:5601`?

---

<div class="post-metadata">

**Author:** ![sgaron\_cse](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sgaron_cse/32/43777_2.png) [@sgaron\_cse](https://discuss.elastic.co/u/sgaron_cse)\
**Post date:** [April 10, 2019, 1:07am UTC](https://discuss.elastic.co/t/apm-viewer-in-kibana-broken-when-transaction-name-has-a-slash-in-it/176044/3 "2019-04-10T01:07:14Z")

</div>

O\_o

You totally nailed it! That was nginx fault...

Just in case someone stumble on this thread:  
If you add a / at the end of the proxy\_pass directive in nginx, links are decode before being passed to the backend server. I removed the slash and now all is good.

proxy\_pass [http://kibana:5601](http://kibana:5601); --\> GOOD

proxy\_pass [http://kibana:5601/](http://kibana:5601/); --\> BAD

Thanks a lot for the help and sorry that you had to test this.

Steve

---

<div class="post-metadata">

**Author:** ![sqren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sqren/32/26110_2.png) [@sqren](https://discuss.elastic.co/u/sqren)\
**Post date:** [April 10, 2019, 12:35pm UTC](https://discuss.elastic.co/t/apm-viewer-in-kibana-broken-when-transaction-name-has-a-slash-in-it/176044/4 "2019-04-10T12:35:22Z")

</div>

Hi Steve,

Glad you found a workaround. And thanks for sharing it with others!  
I'll look into what I can do to mitigate this on our end.

---

<div class="post-metadata">

**Author:** ![sgaron\_cse](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sgaron_cse/32/43777_2.png) [@sgaron\_cse](https://discuss.elastic.co/u/sgaron_cse)\
**Post date:** [April 10, 2019, 7:09pm UTC](https://discuss.elastic.co/t/apm-viewer-in-kibana-broken-when-transaction-name-has-a-slash-in-it/176044/5 "2019-04-10T19:09:02Z")

</div>

To be honest, I don't think there is much you can do from your end. This was mainly my fault.

I guess maybe the only thing could be to have the APIs use ~ instead of % but even then that's just fixing a problem that the user's introduced themselves...

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 1, 2019, 3:09pm UTC](https://discuss.elastic.co/t/apm-viewer-in-kibana-broken-when-transaction-name-has-a-slash-in-it/176044/6 "2019-05-01T15:09:09Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.