I have a transform that aggregates my stats index data for long-term storage, but the results are no longer correct. My stats index stores each request (one document = one request) and stores the credits used by that request, e.g. 6. I transform this into one-day aggregations, e.g. customer A did 1…

You specified a query filter that limits transform to only access data from the last 3 hours, actually even just 1.5 hours, because of the 1.5 hour delay: "must": [{ "range" : {"timestamp": {"gte" : "now-3h"}} }] Such a filter is never a good idea. Transform tak…

Transform missing data

Elastic Stack Elasticsearch

rokcarl (Rok) July 14, 2022, 2:12pm 3

Thank you, that was it and it fixed my problem.

I'm facing a new one though about data mismatch, but I opened a new thread for that.

Topic		Replies	Views
Transform data mismatch with source index Elasticsearch transforms	3	523	August 12, 2022
Transforms: do I need to filter source for time-series data? Elasticsearch transforms	10	1123	July 16, 2021
Transformed index is missing data Elasticsearch transforms	4	554	March 23, 2021
Transforms updates fields from data that exists from before even a filter condition was not met Elasticsearch transforms	4	471	December 23, 2020
Is there a soon update for transforms API Elasticsearch transforms	6	510	November 4, 2021

Transform missing data

Related topics