# Can I set \_ttl in a Logstash config file? **URL:** \ **Category:** Logstash\ **Created:** [October 15, 2015, 6:35pm UTC](https://discuss.elastic.co/t/can-i-set--ttl-in-a-logstash-config-file/32308 "2015-10-15T18:35:22Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![CraigFoote](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/craigfoote/32/4571_2.png) [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)\ **Post date:** [October 15, 2015, 6:35pm UTC](https://discuss.elastic.co/t/can-i-set--ttl-in-a-logstash-config-file/32308/1 "2015-10-15T18:35:22Z")
I see lots of documentation on setting the TTL in a template file to be applied to all indices but can I set them individually? I tried using add\_field for _ttl but got a configtest error saying "_"-prefixed fields cannot be added. ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [October 15, 2015, 6:57pm UTC](https://discuss.elastic.co/t/can-i-set--ttl-in-a-logstash-config-file/32308/2 "2015-10-15T18:57:39Z")
I highly recommend not using TTL for time-series data, like logs. There are good reasons not to pursue this course, like those quoted from this [discussion](https://discuss.elastic.co/t/what-is-the-definitive-way-of-only-retaining-7-days-of-logs/28399/5): > TTLs at first seem like a good idea. "Oh! I can just set this up and it will auto-prune when it hits the pre-defined TTL." The reality is that while this works, it is a Really Bad Idea™ with time-series data, where you know it will always expire in a predictable way. > TTLs force Elasticsearch to check _every single document,_ every 60 seconds (an editable default, but the principle remains). If I have 1,000,000,000 records per day, then I have as many as 1,000,000,000 documents TTLs being checked every 60 seconds, with a 1 day TTL. You can imagine the strain that puts on the disk subsystem, not to mention the hit it would be to queries. On top of this, a TTL-deleted document is _not_ immediately deleted. It is _marked_ for deletion (yep, another I/O operation), and then the delete happens at the next segment merge. Segment merges will, of necessity, be very frequent because of TTLs, which adds to the disk I/O strain. Even if I configure the TTL check to be less frequent (hourly, or even daily), I will still have 1,000,000,000 "mark for deletion" operations, followed immediately by a kajillion segment merges. Oh, and you don't get to choose _when_ the first TTL check happens, so it could be during high use times. > On the other hand, deleting an _entire index_ at once with the index delete API (which is what Curator uses), eliminates every document in a few seconds (because it deletes at the index level), with no more segment merges or disk I/O pain than that. > If you were to compare these two models to SQL commands, the first (TTLs) would be like: > ``` > DELETE FROM TABLE WHERE TIMESTAMP < now-24h; > > ``` > and the second model would be like: > ``` > DROP TABLE TABLENAME; > > ``` > You can see that the first is going to be millions of atomic operations, while the second just drops the entire table. That's what deleting an index vs. TTLs is like, and why TTLs are a Really Bad Idea™ for time-series data. For deleting time-series indices in the recommended way, there's [Elasticsearch Curator](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html). ---
**Author:** ![CraigFoote](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/craigfoote/32/4571_2.png) [@CraigFoote](https://discuss.elastic.co/u/CraigFoote)\ **Post date:** [October 19, 2015, 5:01pm UTC](https://discuss.elastic.co/t/can-i-set--ttl-in-a-logstash-config-file/32308/3 "2015-10-19T17:01:20Z")
Thanks Aaron, good advice. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:26am UTC](https://discuss.elastic.co/t/can-i-set--ttl-in-a-logstash-config-file/32308/4 "2017-07-06T05:26:08Z")