New Index with latest timestamp

Elastic Stack Elasticsearch
1

Hi All,

I have an index that contains time series data, essentially the status of different servers. I want to create another index from this index which should always have the latest value of data present by timestamp, for example, if the polling interval is 5 minutes and I have data with timestamps 18:00, 18:05, 18:10, then the new index must have the documents with the timestamp of 18:10. As soon as the new data arrives at 18:15, the document in the new index must get replaced by the document with 18:15 timestamp. What is a good way to achieve this within the Elastic ecosystem? Any suggestions

2

Hi @Ankita_Pachauri

A latest transform is what you are describing.. exactly for your use case

This is what I was going to suggest in your uptime topic... You could use this to always have the latest state of each monitor.

You will create a latest transform on monitor.id and use the event timestamp to drive the latest qualification.

You can step through this in the Kibana UI... It is pretty simple.

The it is very easy to build the Viz / metrics that you want

1 Like
3

Thanks, it worked!!

1 Like