# Continuous tranforms stop indexing after initial indexing **URL:** \ **Category:** Elasticsearch\ **Created:** [June 24, 2020, 1:06pm UTC](https://discuss.elastic.co/t/continuous-tranforms-stop-indexing-after-initial-indexing/238473 "2020-06-24T13:06:01Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Germain\_Pavot](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/germain_pavot/32/71011_2.png) [@Germain\_Pavot](https://discuss.elastic.co/u/Germain_Pavot)\ **Post date:** [June 24, 2020, 1:06pm UTC](https://discuss.elastic.co/t/continuous-tranforms-stop-indexing-after-initial-indexing/238473/1 "2020-06-24T13:06:01Z")
Hy guys, I try to setup a continuous transforms strategy on a new cluster for optimize queries and index size with an 1h aggregation. The stack is setup like that : - 1 index input-[0\_9]{6} with a write alias "input". A lifestyle policy is attached to split the index when it size reach 100Gb. Logstash put datas to the alias index "input". Refresh interval of index is set to 30s. - 1 index aggr-[0\_9]{6} with a write alias "aggr". A lifestyle policy is attached to split the index when it size reach 100Gb. Refresh interval of index is set to 10s. I setup 10 transforms for various datas with source "input-\*" and destination "aggr" and I have setup a pivot group by date\_histogram on field @timestamp with 1h and the refresh sync is linked to @timestamp with a delay setup to 60s. So after create all of my transforms I start all of them. At this point I identify 2 phases on transforms : - indexing state : transforms indexes all existing datas on indexes "input-\*" - started state : transforms is entered on continuous mode and check new data So on indexing state phase it have no problem, all transforms works and I can see datas on indexes "aggr-\*". But when transforms passed on "started" state no more datas are added... No error, and transforms just stay on "Finished indexing for transform checkpoint [0]." of the previous state. I'am on elasticsearch 7.8.0. It's a bug ? What's can be wrong on my setup ? ---
**Author:** ![Germain\_Pavot](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/germain_pavot/32/71011_2.png) [@Germain\_Pavot](https://discuss.elastic.co/u/Germain_Pavot)\ **Post date:** [June 24, 2020, 2:41pm UTC](https://discuss.elastic.co/t/continuous-tranforms-stop-indexing-after-initial-indexing/238473/2 "2020-06-24T14:41:48Z")
Ok I think I understand, the checkpoint use the current date to compare data ? If this is that, this make sens that it's not working when dates on document are older... ---
**Author:** ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)\ **Post date:** [June 24, 2020, 7:19pm UTC](https://discuss.elastic.co/t/continuous-tranforms-stop-indexing-after-initial-indexing/238473/3 "2020-06-24T19:19:06Z")
That's correct. Continuous mode requires a timestamp that is real time-ish. The source must be append-only, updates of existing documents is not supported. ---
**Author:** ![Germain\_Pavot](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/germain_pavot/32/71011_2.png) [@Germain\_Pavot](https://discuss.elastic.co/u/Germain_Pavot)\ **Post date:** [July 3, 2020, 9:39am UTC](https://discuss.elastic.co/t/continuous-tranforms-stop-indexing-after-initial-indexing/238473/4 "2020-07-03T09:39:29Z")
Yeah, fixed by adding "current\_date" field on records from logstash ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 31, 2020, 9:39am UTC](https://discuss.elastic.co/t/continuous-tranforms-stop-indexing-after-initial-indexing/238473/5 "2020-07-31T09:39:30Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.