# Avoid recalculation from scratch of Transform aggregation

**URL:** https://discuss.elastic.co/t/avoid-recalculation-from-scratch-of-transform-aggregation/218845
**Category:** Elasticsearch
**Created:** [February 11, 2020, 6:54pm UTC](https://discuss.elastic.co/t/avoid-recalculation-from-scratch-of-transform-aggregation/218845 "2020-02-11T18:54:05Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![pmagne](https://avatars.discourse-cdn.com/v4/letter/p/6a8cbe/32.png) [@pmagne](https://discuss.elastic.co/u/pmagne)
#### Post date: [February 11, 2020, 6:54pm UTC](https://discuss.elastic.co/t/avoid-recalculation-from-scratch-of-transform-aggregation/218845/1 "2020-02-11T18:54:06Z")

</div>

Hi,

In a continuous transform, when a new document is added to the source index and the transform has to update one of its own document, it seems that the aggregations calculations are restarted from scratch. It induces that if some documents previously selected by the transform are no longer present in the source index, they are no longer taken into account for the calculations of the aggregations and their information is lost.

Is there a way to tell the transform to only update the aggregation fields based on the new retrieved documents by keeping the previously calculated information intact ?

For instance, in a value\_count aggregation, the previously calculated count would be incremented with the only consideration of the new documents.

---

<div class="post-metadata">

### Author: ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)
#### Post date: [February 12, 2020, 10:24am UTC](https://discuss.elastic.co/t/avoid-recalculation-from-scratch-of-transform-aggregation/218845/2 "2020-02-12T10:24:03Z")

</div>

Hi,

thank you for your feedback. Updating is currently not possible but this is on our list for the future. However the aggregation is not restarted from scratch, but only the changed entities are updated. Still, transform requires that the source does not get deleted.

If you look for compaction, rollup might be the better tool for you.

To give you a bit more background: Updating is simple for count, min, max, a bit more complicated for avg, very complex for e.g. cardinality, percentile. For anything that requires scripts we would need a user supplied merge/combine method. With other words, this is harder than it seems.

Still, there are usecases like yours where updating would be beneficial. Another usecase might be performance related. For large amounts of data, an update should be more performant than the rewrite at the moment.

---

<div class="post-metadata">

### Author: ![pmagne](https://avatars.discourse-cdn.com/v4/letter/p/6a8cbe/32.png) [@pmagne](https://discuss.elastic.co/u/pmagne)
#### Post date: [February 14, 2020, 7:14am UTC](https://discuss.elastic.co/t/avoid-recalculation-from-scratch-of-transform-aggregation/218845/3 "2020-02-14T07:14:33Z")

</div>

Hi,

Thank you for your very clear answer. Indeed, I see now the complexity of the matter for some aggregations.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 13, 2020, 7:14am UTC](https://discuss.elastic.co/t/avoid-recalculation-from-scratch-of-transform-aggregation/218845/4 "2020-03-13T07:14:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
