How can i insert the history data into TSDB

A tsdb data stream is hard coded to set the index.time_series.start_time and index.time_series.end_time settings upon creation of the a backing index.

When a data stream is newly created, the first backing index's start_time setting will be set to: now-look_ahead_time (now-3h in your case) and the end_time setting will be set to: now+look_ahead_time (now+3h in your case). Only data that falls into that range is accepted.

So in your example you will need to adjust the @timestamp field value that fall into that range in order to get these documents accepted. I think the docs should be improved to highlight this fact.

A tsdb data stream is designed to accept recent metric data (which is based on current time and the index.look_ahead_time index setting). As a tsdb data stream gets older and has rolled over then the older backing indices can still accept writes via a data stream (unlike regular data streams). But the most recent backing index is designed to handle recent metric data. The best way to see for what time range a data stream will accept writes, it is best the check the get data stream api. This returns a tsdb data stream's temporal range.

If you're just curious about how well tsdb works for your metric data set, I think it is best to just create a tsdb index (an index with index.mode=time_series) and then experiment how well your data compresses.

1 Like