I just started using transform in kibana UI to aggregate data and also to have a longer retention period for visualization. The data collections work fine, my transform basically collectors two fields which are aggregated on max.
My transform json is
The issue is that the my data collections happens at 7 in the evening but i want that to happen at midnight. Also since i am having a timestamp for 24h because i want to know the max for both fields each day till the previous day.
For example: March 3 12:05 AM should show run the transform and update till the march March 2 day .
How could i configure the transform to perform on a schedule time everyday to get the daily peaks of the two fields till the previous day. Currently my transform run every day at evening 7 and it generates till the previous day till the same time.
Just want to schedule the time to early morning at 12:10 from 7PM.
Your transform runs every hour, because
frequency is set to
1h. What you are experiencing is not a problem of scheduling, but a timezone issue. Elasticsearch uses the UTC time, to have a consistent time allover the world. All dates are stored in UTC and the date_histogram will produce buckets according to UTC. Transform now runs shortly after midnight, but it is midnight according to UTC, not midnight according to your timezone.
You can change the timezone of your
date_histogram. This will tell the aggregation to create buckets according to the specified timezone.
Transform will still check every hour, but now start shortly after midnight according to the configured timezone.
FWIW: The behavior is by design, if a
date_histogram is configured transform executes - at default - when it retrieved all data for a bucket. You can change the default in
settings by setting
false. This will produce intermediate results, so you can see the data of the current day while the day isn't over yet. However that is obviously more expensive as it re-calculates the buckets more often. That's why
align_checkpoints has the default setting
Thanks @Hendrik_Muhs for the answer. One more thing, i wanted to ask is when i am visualizing the data of the transform in a line lens. I have two fields in group_by block devndevice and devname, I need to display two lines for same aggregation but from different devname in same visualization using kibana. Currently its calculating the max of all the the same field across all devname and visualizing a single line. Is there any way that i can get multiple line with the same field with different devname.
FOr example: we have devname=test1 devndevice=test, and devname=test2 devndevice=test, I should be able to visualize both the metrics for different devndevice not just the single one that is combining both devndeivce.
I am sorry, I can't answer the last question. From a data perspective it should work, because you group by
devname in addition to
I hope someone else can answer. If you should not get an answer after some waiting, please open a new thread.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.