# Old metrics of APM dont show when upgrading 7.17 to 8.9 **URL:** \ **Category:** APM\ **Tags:** rum, java, server, ui\ **Created:** [September 26, 2023, 8:24pm UTC](https://discuss.elastic.co/t/old-metrics-of-apm-dont-show-when-upgrading-7-17-to-8-9/343927 "2023-09-26T20:24:34Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![ElasticLiver](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elasticliver/32/64869_2.png) [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Post date:** [September 26, 2023, 8:24pm UTC](https://discuss.elastic.co/t/old-metrics-of-apm-dont-show-when-upgrading-7-17-to-8-9/343927/1 "2023-09-26T20:24:34Z")
Hi, I was tasked to upgrade elastic, kibana, logstash, and APM Server from 7.17 to 8.9, when I did the upgrade, old metrics from APM do not show in kibana APM section, only the new ones, so we restore the snapshot of the old stack. We are using java agent and RUM the APM server send data to logstash and then to elastic. What could be wrong? ---
**Author:** ![adsandie](https://avatars.discourse-cdn.com/v4/letter/a/8c91f0/32.png) [@adsandie](https://discuss.elastic.co/u/adsandie)\ **Post date:** [September 26, 2023, 10:52pm UTC](https://discuss.elastic.co/t/old-metrics-of-apm-dont-show-when-upgrading-7-17-to-8-9/343927/2 "2023-09-26T22:52:13Z")
I also have this problem. Upon investigation, it seems that the 8.9.1 is using data stream while the old 7.x.x version are not using the new fields. On the 8.9.1, there is a new field metricset.samples.name regarding jvm, transactions, etc and metricset.samples.value. The APM Dashboard is still using the old 7.x.x settings. ---
**Author:** ![adsandie](https://avatars.discourse-cdn.com/v4/letter/a/8c91f0/32.png) [@adsandie](https://discuss.elastic.co/u/adsandie)\ **Post date:** [September 27, 2023, 12:40am UTC](https://discuss.elastic.co/t/old-metrics-of-apm-dont-show-when-upgrading-7-17-to-8-9/343927/3 "2023-09-27T00:40:24Z")
The only solution we found was still use the 7.17.13 apm-server. The problem was the 8.x and might be the incorrect mapping. ---
**Author:** ![adsandie](https://avatars.discourse-cdn.com/v4/letter/a/8c91f0/32.png) [@adsandie](https://discuss.elastic.co/u/adsandie)\ **Post date:** [September 27, 2023, 1:19am UTC](https://discuss.elastic.co/t/old-metrics-of-apm-dont-show-when-upgrading-7-17-to-8-9/343927/4 "2023-09-27T01:19:05Z")
This is the problem for us. It looks like a workaround maybe for us to create the ingest pipeline and copy the event.duration to [transaction.duration.us](http://transaction.duration.us) ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/a/3aeda7537f51ba048db995ef544704209af9d2ad.png) ---
**Author:** ![Andrew\_Mora](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_mora/32/125622_2.png) [@Andrew\_Mora](https://discuss.elastic.co/u/Andrew_Mora)\ **Post date:** [September 27, 2023, 12:16pm UTC](https://discuss.elastic.co/t/old-metrics-of-apm-dont-show-when-upgrading-7-17-to-8-9/343927/5 "2023-09-27T12:16:05Z")
> [@ElasticLiver](#): > > Hi, I was tasked to upgrade elastic, kibana, logstash, and APM Server from 7.17 to 8.9, when I did the upgrade, old metrics from APM do not show in kibana APM section, only the new ones, so we restore the snapshot of the old stack. > > We are using java agent and RUM > > the APM server send data to logstash and then to elastic. > > What could be wrong? It's crucial to ensure compatibility between your Java agent, RUM, APM Server, and Elasticsearch versions when upgrading. Investigate version compatibility and configuration settings to resolve the issue and retain access to historical APM metrics in Kibana. Seek support or documentation from Elastic for detailed guidance. Best of luck with your upgrade! AC Football Cases. ---
**Author:** ![Carson\_Ip](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carson_ip/32/115912_2.png) [@Carson\_Ip](https://discuss.elastic.co/u/Carson_Ip)\ **Post date:** [September 28, 2023, 2:53pm UTC](https://discuss.elastic.co/t/old-metrics-of-apm-dont-show-when-upgrading-7-17-to-8-9/343927/6 "2023-09-28T14:53:11Z")
Hi @ElasticLiver and @adsandie , I've managed to reproduce the issue where incomplete data is shown on APM UI after upgrade. Please be assured that there is no data loss as this is just a UI issue. This happens when the APM UI queries for a time range that covers the upgrade window, when both old and new data exists. The root cause is that there are a lot of performance improvements in 8.7 apm-server ([see changelog](https://www.elastic.co/guide/en/apm/guide/current/release-notes-8.7.html#_added_4)), with a major theme around "service metrics", which are pre-aggregated metrics to speed up APM UI (see [blog post](https://www.elastic.co/blog/improving-elastic-apm-ui-performance-continuous-rollups-service-metrics)). An unfortunate side effect of introducing these new data streams is that when data (new data ingested since 8.9 upgrade) in the new pre-aggregated data streams exists, the UI returns immediately and do not fallback to query the old data streams (where your 7.17 data are). When the query only covers time before upgrade, because there is no data in new data streams, then the old data will show up. The above implication will be documented as part of [[APM] Document the UI query strategies for fetching data · Issue #155066 · elastic/kibana · GitHub](https://github.com/elastic/kibana/issues/155066) A workaround to potentially fix the UI weirdness is to disable service metrics under APM settings in Kibana. But there may be some graphs that are not affected by this feature toggle. To reiterate, there is no data loss and as long as you don't query across the upgrade time, the UI should perform as expected. This also means that it will be lesser of an issue the longer you use the newer version. ---
**Author:** ![adsandie](https://avatars.discourse-cdn.com/v4/letter/a/8c91f0/32.png) [@adsandie](https://discuss.elastic.co/u/adsandie)\ **Post date:** [September 28, 2023, 11:55pm UTC](https://discuss.elastic.co/t/old-metrics-of-apm-dont-show-when-upgrading-7-17-to-8-9/343927/7 "2023-09-28T23:55:35Z")
I resolved this issue by creating the ingest pipeline and copying the value from event.duration to [transaction.duration.us](http://transaction.duration.us) and [span.duration.us](http://span.duration.us) and using 8.8.2 version. The 8.9.1 is another issue and needed time for us to troubleshoot further. RIght now, elasticsearch, kibana, logstash, beats are using 8.9.1 and apm-server is using 8.8.2. This is an upgrade from 7.17.x to 8.x ---
**Author:** ![ElasticLiver](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/elasticliver/32/64869_2.png) [@ElasticLiver](https://discuss.elastic.co/u/ElasticLiver)\ **Post date:** [October 25, 2023, 11:19pm UTC](https://discuss.elastic.co/t/old-metrics-of-apm-dont-show-when-upgrading-7-17-to-8-9/343927/8 "2023-10-25T23:19:47Z")
I was thinking to create a dashboard with some metrics of APM, So if the client want to view the old data has to visit the dashboard, and to view the new data has to use the the APM in the observability section, is this possible? ---
**Author:** ![kyungeunni](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kyungeunni/32/125539_2.png) [@kyungeunni](https://discuss.elastic.co/u/kyungeunni)\ **Post date:** [October 26, 2023, 3:57am UTC](https://discuss.elastic.co/t/old-metrics-of-apm-dont-show-when-upgrading-7-17-to-8-9/343927/9 "2023-10-26T03:57:21Z")
Hi @ElasticLiver, It would require discipline in using date range filters for the dashboards and the APM UI, as there are no means to restrict the time range per page. You can share the links to the Dashboards with a specified time range though, and warn the clients that they won't see some data if they change the time range filters. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 23, 2023, 3:58am UTC](https://discuss.elastic.co/t/old-metrics-of-apm-dont-show-when-upgrading-7-17-to-8-9/343927/10 "2023-11-23T03:58:06Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.