# Data frame vs datafeed vs rollup
**URL:** https://discuss.elastic.co/t/data-frame-vs-datafeed-vs-rollup/213811
**Category:** Elasticsearch
**Tags:** elastic-stack-machine-learning
**Created:** [January 5, 2020, 4:54pm UTC](https://discuss.elastic.co/t/data-frame-vs-datafeed-vs-rollup/213811 "2020-01-05T16:54:44Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@liorg2](https://discuss.elastic.co/u/liorg2)
#### Post date: [January 5, 2020, 4:54pm UTC](https://discuss.elastic.co/t/data-frame-vs-datafeed-vs-rollup/213811/1 "2020-01-05T16:54:44Z")
Hello
There are few components in the Elastic stack, which I'm not sure I understand the difference and when to use which.
for ML I use (currently) datafeed with pre-defined query (in practice it seems to aggregate metrics by a given time frame)
Rollup adds a functionality for aggregate raw indices and save query time/ space..
And in the latest version, after upgrading from 6.7 to 7.5, there is a new feature called 'data frames', which sounds similar..
So.. if my purpose is to aggregate data into smaller indices which can accept queries, and also run ML jobs. which should I choose, and why there are few features with similar functionality?
thanks
Lior
---
### Author:  [@edsavage](https://discuss.elastic.co/u/edsavage)
#### Post date: [January 6, 2020, 2:57pm UTC](https://discuss.elastic.co/t/data-frame-vs-datafeed-vs-rollup/213811/2 "2020-01-06T14:57:07Z")
Hi Lior,
All good questions! I will do my best to answer them in detail. Please bear with me while I formulate my response (with a little help from my team).
Kind Regards,
Ed
---
### Author:  [@grabowskit](https://discuss.elastic.co/u/grabowskit)
#### Post date: [January 7, 2020, 7:37pm UTC](https://discuss.elastic.co/t/data-frame-vs-datafeed-vs-rollup/213811/3 "2020-01-07T19:37:46Z")
Lior,
I believe what you want is [Transforms](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html) which allow you to convert existing Elasticsearch indices into summarized indices.
The other functionality you mention is very use case specific and probably won't do what you need based on your stated purpose. [Datafeeds](https://www.elastic.co/guide/en/machine-learning/7.5/ml-dfeeds.html) are only used for feeding data into ML anomaly detection jobs (and don't create other indices). [Rollups](https://www.elastic.co/guide/en/kibana/current/data-rollups.html) are used to aggregate metric indices to reduce storage, but have a special \_search endpoint that allows you to query across raw and summarized metric data.
Thanks for the question,
TomG
---
### Author:  [@liorg2](https://discuss.elastic.co/u/liorg2)
#### Post date: [January 7, 2020, 10:19pm UTC](https://discuss.elastic.co/t/data-frame-vs-datafeed-vs-rollup/213811/4 "2020-01-07T22:19:15Z")
Thanks a lot!
The issue with transforms (for me), is that I use percentiles aggregation often, and it seems to be missing from current supported aggregations.
---
### Author:  [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [January 8, 2020, 11:08am UTC](https://discuss.elastic.co/t/data-frame-vs-datafeed-vs-rollup/213811/5 "2020-01-08T11:08:24Z")
> [@liorg2](#):
>
> why there are few features with similar functionality?
On this specific point - roll ups are a form of compaction that is geared specifically towards grouping documents based on time units while transforms are typically grouping documents on a choice of entity like a customer ID.
A web session is an example of an entity that can span time units so time-based roll-ups are not an appropriate mechanism for grouping that information.
A monthly roll-up is not grouped on a single entity key (it's a range of timestamps) so transforms are not appropriate.
What you might want to summarise for entities vs time-groups might be similar (counts, flags etc) but the unit that you group things around is fundamentally different.
---
### Author:  [@liorg2](https://discuss.elastic.co/u/liorg2)
#### Post date: [January 9, 2020, 7:46am UTC](https://discuss.elastic.co/t/data-frame-vs-datafeed-vs-rollup/213811/6 "2020-01-09T07:46:48Z")
Do you guys know if percentiles are in the road map of transforms?
thanks
@Mark_Harwood
@grabowskit
@edsavage
---
### Author:  [@sophie\_chang](https://discuss.elastic.co/u/sophie_chang)
#### Post date: [January 9, 2020, 9:43am UTC](https://discuss.elastic.co/t/data-frame-vs-datafeed-vs-rollup/213811/7 "2020-01-09T09:43:56Z")
Yes, percentiles are on our roadmap including handling for functions that return multiple values. It is something we are keen to do, however we do not have committed timeframes for this yet.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 6, 2020, 9:44am UTC](https://discuss.elastic.co/t/data-frame-vs-datafeed-vs-rollup/213811/8 "2020-02-06T09:44:07Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.