# Rollups of APM high level data

**URL:** <https://discuss.elastic.co/t/rollups-of-apm-high-level-data/253447>\
**Category:** APM\
**Tags:** ilm-index-lifecycle-management, server, ui\
**Created:** [October 27, 2020, 2:07pm UTC](https://discuss.elastic.co/t/rollups-of-apm-high-level-data/253447 "2020-10-27T14:07:54Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jkaoiadjmdna](https://avatars.discourse-cdn.com/v4/letter/j/858c86/32.png) [@jkaoiadjmdna](https://discuss.elastic.co/u/jkaoiadjmdna)\
**Post date:** [October 27, 2020, 2:07pm UTC](https://discuss.elastic.co/t/rollups-of-apm-high-level-data/253447/1 "2020-10-27T14:07:54Z")

</div>

I'm working on a strategy for rolling up APM data. Specifically, things like detailed errors, transactions, spans are not much use after some period of time. However, simplified data like requests per minute, and transaction duration can be interesting for us over a long time.

What do I need to do to only maintain these metrics? In one sense, what I want is for the Transaction Duration and Requests per minute charts in APM to work for all-time. It's fine if I need to rebuild those charts to also include historical data but that's my end goal.

---

<div class="post-metadata">

**Author:** ![dgieselaar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dgieselaar/32/46947_2.png) [@dgieselaar](https://discuss.elastic.co/u/dgieselaar)\
**Post date:** [October 28, 2020, 9:25am UTC](https://discuss.elastic.co/t/rollups-of-apm-high-level-data/253447/2 "2020-10-28T09:25:53Z")

</div>

Hello @jkaoiadjmdna! This is not super straight-forward right now. We are working on various initiatives to make this easier in the future. Right now, I can think of two options:

- You only need transaction events for the Transaction Duration and Requests/Transactions per minute to work. If you want to save storage, you could drop fields from (older) transaction events. This could be done via a processor. You'd at least need `service.name`, `service.environment`, `transaction.name`, `transaction.type`, `parent.id`, `processor.event`, and `@timestamp`. Depending on your needs, some other fields might be needed as well. You could delete spans, errors, and metrics with [ILM](https://www.elastic.co/guide/en/apm/server/current/ilm-reference.html).
- If you are willing to rebuild charts outside of the APM app, for instance by using Visualize, you could try to leverage [Elasticsearch transforms](https://www.elastic.co/guide/en/elasticsearch/reference/current/transform-apis.html). In this case, you'd need to group by the aforementioned fields (except for `parent.id`), and add a `value_count` and `sum` aggregation. You can then use a `weighted_average` aggregation to get the transaction duration, and a `rate` aggregation to get the transactions per minute.

In the next release, 7.10, we are also shipping an experimental feature that reduces storage cost of transaction duration data significantly. [We are currently working on documentation for APM Server](https://github.com/elastic/apm-server/pull/4345/files). You'd also need to enable it in Kibana by setting `xpack.apm.searchAggregatedTransactions` to `"always"` in your Kibana configuration.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 18, 2020, 5:26am UTC](https://discuss.elastic.co/t/rollups-of-apm-high-level-data/253447/3 "2020-11-18T05:26:06Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.