# APM transactions not showing in Kibana after upgrade from 7.5.0 to 7.5.2
**URL:** https://discuss.elastic.co/t/apm-transactions-not-showing-in-kibana-after-upgrade-from-7-5-0-to-7-5-2/219269
**Category:** APM
**Tags:** server
**Created:** [February 13, 2020, 5:57pm UTC](https://discuss.elastic.co/t/apm-transactions-not-showing-in-kibana-after-upgrade-from-7-5-0-to-7-5-2/219269 "2020-02-13T17:57:48Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@cotjoey](https://discuss.elastic.co/u/cotjoey)
#### Post date: [February 13, 2020, 5:57pm UTC](https://discuss.elastic.co/t/apm-transactions-not-showing-in-kibana-after-upgrade-from-7-5-0-to-7-5-2/219269/1 "2020-02-13T17:57:49Z")
This is to troubleshoot this issue in GitHub - You can find more information here: [https://github.com/elastic/apm/issues/211](https://github.com/elastic/apm/issues/211)
**Kibana version** : 7.5.2
**Elasticsearch version** : 7.5.2
**APM Server version** : 7.5.2
**APM Agent language and version** : elastic-apm-agent-1.12.0.jar
**Browser version** : Google Chrome Version 80.0.3987.87 (Official Build) (64-bit)
**Original install method (e.g. download page, yum, deb, from source, etc.) and version**: yum install apm-server [From 7.x repository]
**Fresh install or upgraded from other version?** : Upgraded APM from 7.5.0 to 7.5.2
**Is there anything special in your setup?**
- I use Logstash to send events to from APM;
- I have a load balancer in front of APM;
- I did not change any configuration. At 7.5.0 I was seeing events in the APM app in Kibana, after the upgrade to 7.5.2 I wasn't seeing anything.
- One thing I tried is to run setup from APM server to create the index templates to try to resolve the issue - it didn't work.
**Description of the problem including expected versus actual behavior. Please include screenshots (if relevant)**:
**Steps to reproduce** :
1. yum install -y apm-server
2. systemctl restart apm-server
**Errors in browser console (if relevant)**:
I do not see errors, but those three API calls take multiple seconds to finish:
[https://kibana.example.com/api/apm/ui\_filters/environments?start=2020-02-12T17%3A47%3A44.903Z&end=2020-02-13T17%3A47%3A44.904Z](https://kibana.example.com/api/apm/ui_filters/environments?start=2020-02-12T17%3A47%3A44.903Z&end=2020-02-13T17%3A47%3A44.904Z)
[https://kibana.example.com/api/apm/ui\_filters/local\_filters/services?uiFilters={}&start=2020-02-12T17%3A47%3A44.903Z&end=2020-02-13T17%3A47%3A44.904Z&filterNames=["host"%2C"agentName"]](https://kibana.example.com/api/apm/ui_filters/local_filters/services?uiFilters=%7B%7D&start=2020-02-12T17%3A47%3A44.903Z&end=2020-02-13T17%3A47%3A44.904Z&filterNames=%5B%22host%22%2C%22agentName%22%5D)
[https://kibana.example.com/api/apm/services?start=2020-02-12T17%3A47%3A44.903Z&end=2020-02-13T17%3A47%3A44.904Z&uiFilters={}](https://kibana.example.com/api/apm/services?start=2020-02-12T17%3A47%3A44.903Z&end=2020-02-13T17%3A47%3A44.904Z&uiFilters=%7B%7D)
**Provide logs and/or server output (if relevant)**:
- The agent log shows everything being fine (nothing changed here anyway):
```auto
2020-02-10 10:53:35.579 [apm-server-healthcheck] INFO co.elastic.apm.agent.report.ApmServerHealthChecker - Elastic APM server is available: { "build_date": "2020-01-15T10:22:43Z", "build_sha": "0683f8094f42fecedd54cee09c0991febd8ce59a", "version": "7.5.2"}
2020-02-10 10:53:35.668 [apm-remote-config-poller] INFO co.elastic.apm.agent.configuration.ApmServerConfigurationSource - Received new configuration from APM Server: {}
```
- In the Elasticsearch logs, I see the following events being repeated over and over (with a bunch of active gc.log):
```auto
[2020-02-12T21:32:28,003][DEBUG][o.e.a.s.TransportSearchAction] [master-node-1] [apm-7.5.2][0], node[FYBwRUMpTpm2yZs6FPNnXg], [P], s[STARTED], a[id=MaHC49h9RamInCFnwUuyEA]: Failed to execute [SearchRequest{searchType=QUERY_THEN_FETCH, indices=[apm-*, apm-*, apm-*], indicesOptions=IndicesOptions[ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=true, ignore_aliases=false, ignore_throttled=true], types=[], routing='null', preference='null', requestCache=null, scroll=null, maxConcurrentShardRequests=0, batchedReduceSize=512, preFilterShardSize=128, allowPartialSearchResults=true, localClusterAlias=null, getOrCreateAbsoluteStartMillis=-1, ccsMinimizeRoundtrips=true, source={"size":0,"query":{"bool":{"filter":[{"terms":{"processor.event":["transaction","error","metric"],"boost":1.0}},{"range":{"@timestamp":{"from":1581471148098,"to":1581557548098,"include_lower":true,"include_upper":true,"format":"epoch_millis","boost":1.0}}},{"range":{"observer.version_major":{"from":7,"to":null,"include_lower":true,"include_upper":true,"boost":1.0}}}],"adjust_pure_negative":true,"boost":1.0}},"aggregations":{"environments":{"terms":{"field":"service.environment","missing":"ENVIRONMENT_NOT_DEFINED","size":10,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},{"_key":"asc"}]}}}}}]
```
```auto
org.elasticsearch.transport.RemoteTransportException: [master-node-1][172.31.20.22:9300][indices:data/read/search[phase/query]]
Caused by: java.lang.IllegalArgumentException: Fielddata is disabled on text fields by default. Set fielddata=true on [host.hostname] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.
```
```auto
org.elasticsearch.transport.RemoteTransportException: [master-node-1][172.31.20.22:9300][indices:data/read/search[phase/query]]
Caused by: java.lang.IllegalArgumentException: Fielddata is disabled on text fields by default. Set fielddata=true on [service.name] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.
```
```auto
org.elasticsearch.transport.RemoteTransportException: [master-node-1][172.31.20.22:9300][indices:data/read/search[phase/query]]
Caused by: java.lang.IllegalArgumentException: Fielddata is disabled on text fields by default. Set fielddata=true on [service.environment] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.
```
- GET apm-\*/\_mapping/field/service.name yields this in the dev tools (truncated for clarity):
```auto
"apm-7.5.2" : {
"mappings" : {
"service.name" : {
"full_name" : "service.name",
"mapping" : {
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
},
"apm-7.5.0" : {
"mappings" : {
"service.name" : {
"full_name" : "service.name",
"mapping" : {
"name" : {
"type" : "keyword",
"ignore_above" : 1024
}
}
}
}
},
```
---
### Author:  [@jalvz](https://discuss.elastic.co/u/jalvz)
#### Post date: [February 14, 2020, 9:49am UTC](https://discuss.elastic.co/t/apm-transactions-not-showing-in-kibana-after-upgrade-from-7-5-0-to-7-5-2/219269/2 "2020-02-14T09:49:31Z")
Hi there,
Thanks for the info, the 7.5.2 mapping definitely looks wrong - `service.name` type should not be "text".
Can you stop the apm-server, run `apm-server setup -e`, and then start it again and query the mapping once more? That should fix it for you.
Hope it helps,
Juan
---
### Author:  [@cotjoey](https://discuss.elastic.co/u/cotjoey)
#### Post date: [February 18, 2020, 2:29pm UTC](https://discuss.elastic.co/t/apm-transactions-not-showing-in-kibana-after-upgrade-from-7-5-0-to-7-5-2/219269/3 "2020-02-18T14:29:05Z")
Hello Juan,
Thank you for the prompt response. I don't understand clearly how I fixed the issue. Before I saw your update, I was able to fix the issue myself, and those are the steps I followed:
1. Stop APM & Logstash from sending APM events to Elasticsearch;
2. Deleted the "apm-7.5.2" index, index template & index pattern;
3. From the "APM" app in Kibana, under "Setup Instructions" in clicked on "Load Kibana objects";
4. I ran the following setup command:
```auto
apm-server setup -e -E output.logstash.enabled=false -E output.elasticsearch.hosts=['elastic.example.com:9200'] -E setup.kibana.host=kibana.example.com:5601
```
1. Started Logstash & APM back up;
2. My APM metrics were now showing in the Kibana Dashboard.
That leads me to more questions...
1. At first, when I tried to fix the issue, I was trying to run command:
```auto
apm-server setup --index-management -E apm-server.ilm.enabled=true -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["elastic.example.com:9200"]' -c /etc/apm-server/apm-server.yml
```
... how does that command differ from the other one?
1. Why do I get this issue in the first place? The error I reported about was after an upgrade from 7.5.0 to 7.5.2, but I just recently installed a production environment from scratch which installed 7.6.0 directly and I had the same index issue. It's causing a bit of grief when we are trying to automate things and end up realizing that we have to do manual steps after every upgrade.
Thank you for your assistance.
Joey
---
### Author:  [@jalvz](https://discuss.elastic.co/u/jalvz)
#### Post date: [February 21, 2020, 2:25pm UTC](https://discuss.elastic.co/t/apm-transactions-not-showing-in-kibana-after-upgrade-from-7-5-0-to-7-5-2/219269/4 "2020-02-21T14:25:57Z")
Hi,
If you are using Logstash, you always need to run the `setup` command on upgrades and fresh installs, as Logstash doesn't know what template APM needs.
You would need to automate that.
As for the setup subcomand you paste, I am not sure what is wrong ... for what I can see, it should work.
For next time, I recommend always to pass `-e` so you can see if there are any errors.
I hope this helps!
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 13, 2020, 10:25am UTC](https://discuss.elastic.co/t/apm-transactions-not-showing-in-kibana-after-upgrade-from-7-5-0-to-7-5-2/219269/5 "2020-03-13T10:25:59Z")
This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.