# Data expiration and ttl **URL:** \ **Category:** Elasticsearch\ **Created:** [January 9, 2016, 8:02pm UTC](https://discuss.elastic.co/t/data-expiration-and-ttl/38793 "2016-01-09T20:02:20Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![ycunc](https://avatars.discourse-cdn.com/v4/letter/y/dec6dc/32.png) [@ycunc](https://discuss.elastic.co/u/ycunc)\ **Post date:** [January 9, 2016, 8:02pm UTC](https://discuss.elastic.co/t/data-expiration-and-ttl/38793/1 "2016-01-09T20:02:20Z")
Given a stream of data coming in every second, but we only want to keep data in the past T time (say 1 hour). What is the best way to expire and remove old data? We did some research and found the following two 1. Set ttl of each document to T, and ES will automatically black list old data and remove them. One question we have is when and how frequently will the data be physically removed? Is it controlled by indices.ttl.interval or something else? 2. Use time-frame based indexes, and index data every T time frame. However, this approach might introduce very strange tfidf scores for the latest index when it has very few data. Is there a good way to handle this? Thanks! ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 9, 2016, 9:26pm UTC](https://discuss.elastic.co/t/data-expiration-and-ttl/38793/2 "2016-01-09T21:26:49Z")
TTL is deprecated and will be removed in upcoming versions. So you should definitely use time based indices instead. ---
**Author:** ![ycunc](https://avatars.discourse-cdn.com/v4/letter/y/dec6dc/32.png) [@ycunc](https://discuss.elastic.co/u/ycunc)\ **Post date:** [January 9, 2016, 9:48pm UTC](https://discuss.elastic.co/t/data-expiration-and-ttl/38793/3 "2016-01-09T21:48:32Z")
Thanks for the reply. However, for time based indices, is there any helper functions for this? We are concern about a new index with very few data in it. It might have very different tfidf values, and might introduce strange search results. Is there any good way to handle these? Thanks! ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 10, 2016, 1:22am UTC](https://discuss.elastic.co/t/data-expiration-and-ttl/38793/4 "2016-01-10T01:22:28Z")
Not really, it might be worth looking into just deleting documents directly? Or perhaps someone else has other ideas. ---
**Author:** ![ycunc](https://avatars.discourse-cdn.com/v4/letter/y/dec6dc/32.png) [@ycunc](https://discuss.elastic.co/u/ycunc)\ **Post date:** [January 10, 2016, 5:34am UTC](https://discuss.elastic.co/t/data-expiration-and-ttl/38793/5 "2016-01-10T05:34:28Z")
Yes directly issuing a request to delete data is also fine. However, what's the difference between ttl and directly delete? They both will only blacklist deleted items and then remove them during segment merging? any particular reason in favor of directly deleting? or just because ttl is getting deprecated so we prefer delete? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 10, 2016, 9:02am UTC](https://discuss.elastic.co/t/data-expiration-and-ttl/38793/6 "2016-01-10T09:02:40Z")
TTL means constantly scanning the entire index looking for documents to be deleted, which is expensive. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:25pm UTC](https://discuss.elastic.co/t/data-expiration-and-ttl/38793/7 "2017-07-05T23:25:41Z")