# \_ttl problems Bulk deletion failures **URL:** \ **Category:** Elasticsearch\ **Created:** [January 27, 2016, 11:52am UTC](https://discuss.elastic.co/t/-ttl-problems-bulk-deletion-failures/40222 "2016-01-27T11:52:25Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![tswider](https://avatars.discourse-cdn.com/v4/letter/t/f07891/32.png) [@tswider](https://discuss.elastic.co/u/tswider)\ **Post date:** [January 27, 2016, 11:52am UTC](https://discuss.elastic.co/t/-ttl-problems-bulk-deletion-failures/40222/1 "2016-01-27T11:52:25Z")
I have following problem: I have \_ttl set on all the indexes, I have migrated from 1.5 to 1.7 a few weeks ago, the \_ttl was working perfectly in 1.5. And since yesterday I have following errors in the log: " [2016-01-27 11:30:37,361][ERROR][indices.ttl] [elknode2] bulk deletion failures for [1680]/[2111] items" And the cluster is behaving very badly, searches are slow and indexing is slow. I have found in [https://discuss.elastic.co/t/ttl-purge-not-working-after-upgrade-from-1-5-2-to-1-7-1/26787](https://discuss.elastic.co/t/ttl-purge-not-working-after-upgrade-from-1-5-2-to-1-7-1/26787) that this error is related to shield plugin blocking the delete but I do not have the plugin installed. My question is - can I switch of the \_ttl removing so it does not try to remove the expired documents ? I am planing to migrate to day or week based indexes, and delete them with no \_ttl but this migration will not be easy, as indexes are big and in use, I can not just re-index it all, I need to sort of grow the new ones. ---
**Author:** ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy)\ **Post date:** [January 27, 2016, 12:10pm UTC](https://discuss.elastic.co/t/-ttl-problems-bulk-deletion-failures/40222/2 "2016-01-27T12:10:33Z")
Hi, > [@tswider](#): > > My question is - can I switch of the \_ttl removing so it does not try to remove the expired documents ? There is an index setting called `index.ttl.disable_purge`; when set to true, documents won't be deleted by the TTL purge service. I'm curious to have more info and log message about why the deletions failed. ---
**Author:** ![tswider](https://avatars.discourse-cdn.com/v4/letter/t/f07891/32.png) [@tswider](https://discuss.elastic.co/u/tswider)\ **Post date:** [January 27, 2016, 12:42pm UTC](https://discuss.elastic.co/t/-ttl-problems-bulk-deletion-failures/40222/3 "2016-01-27T12:42:33Z")
`[2016-01-27 03:26:44,494][ERROR][indices.ttl] [elknode2] bulk deletion failures for [10000]/[10000] items [2016-01-27 03:26:44,515][ERROR][indices.ttl] [elknode2] bulk deletion failures for [10000]/[10000] items [2016-01-27 03:26:44,590][ERROR][indices.ttl] [elknode2] bulk deletion failures for [10000]/[10000] items [2016-01-27 03:26:44,749][ERROR][indices.ttl] [elknode2] bulk deletion failures for [10000]/[10000] items [2016-01-27 03:26:44,763][ERROR][indices.ttl] [elknode2] bulk deletion failures for [10000]/[10000] items [2016-01-27 03:26:44,785][ERROR][indices.ttl] [elknode2] bulk deletion failures for [10000]/[10000] items [2016-01-27 03:26:44,796][ERROR][indices.ttl] [elknode2] bulk deletion failures for [4382]/[4382] items [2016-01-27 03:26:44,920][ERROR][indices.ttl] [elknode2] bulk deletion failures for [613]/[613] items [2016-01-27 03:26:44,974][ERROR][indices.ttl] [elknode2] bulk deletion failures for [588]/[588] items [2016-01-27 03:26:45,085][ERROR][indices.ttl] [elknode2] bulk deletion failures for [650]/[650] items [2016-01-27 03:26:45,265][ERROR][indices.ttl] [elknode2] bulk deletion failures for [605]/[605] items [2016-01-27 03:26:45,360][ERROR][indices.ttl] [elknode2] bulk deletion failures for [611]/[611] items [2016-01-27 03:26:45,918][ERROR][indices.ttl] [elknode2] bulk deletion failures for [4]/[4] items [2016-01-27 03:26:45,920][ERROR][indices.ttl] [elknode2] bulk deletion failures for [4]/[4] items [2016-01-27 03:26:45,921][ERROR][indices.ttl] [elknode2] bulk deletion failures for [4]/[4] items [2016-01-27 03:26:45,925][ERROR][indices.ttl] [elknode2] bulk deletion failures for [3]/[3] items [2016-01-27 03:26:45,926][ERROR][indices.ttl] [elknode2] bulk deletion failures for [4]/[4] items [2016-01-27 03:26:45,927][ERROR][indices.ttl] [elknode2] bulk deletion failures for [4]/[4] items [2016-01-27 03:27:10,659][ERROR][indices.ttl] [elknode2] bulk deletion failures for [7313]/[7313] items [2016-01-27 03:27:12,115][ERROR][indices.ttl] [elknode2] bulk deletion failures for [7249]/[7249] items [2016-01-27 03:27:13,909][ERROR][indices.ttl] [elknode2] bulk deletion failures for [7257]/[7257] items [2016-01-27 03:27:15,514][ERROR][indices.ttl] [elknode2] bulk deletion failures for [7301]/[7301] items` Here is some more log, it is not saying why 🙂 it is failing. But thanks for the setting tip for index.ttl.disable\_purge, it seems to be working, after setting that the cluster is way more responsive. I might be wrong but it looks like indexing is two times faster than yesterday. I have cluster with 2 nodes of 1.7.0 and 43 indexes with different \_tt times on them. these are various event logs and I need to keep 30 days worth of it. Some of the \_ttl purging was still working (I could see that total count of documents in kopf plugin decreased) so the purging was working for some indexes but not all of them. I do not know why it was failing but I am motivated now to remove the \_ttl and use dated indexes. ---
**Author:** ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy)\ **Post date:** [January 27, 2016, 12:49pm UTC](https://discuss.elastic.co/t/-ttl-problems-bulk-deletion-failures/40222/4 "2016-01-27T12:49:51Z")
> [@tswider](#): > > I do not know why it was failing but I am motivated now to remove the \_ttl and use dated indexes. That's definitely the way to go and you should have better performance: TTL purge service executes bulk deletes of documents where deleting timestamped indices is basically freeing resources and deleting files. Do you have read-only indices? Do your documents continuously updated in background? Bulk deletions often failed because the document has been updated in the meanwhile. You can also turn on TRACE logging for `indices.ttl` to know more about the errors but be careful because that will be very very verbose... Maybe you can snapshot some indices and restore them on a testing cluster and see if you can reproduce the issue and then enable TRACE logging. ---
**Author:** ![tswider](https://avatars.discourse-cdn.com/v4/letter/t/f07891/32.png) [@tswider](https://discuss.elastic.co/u/tswider)\ **Post date:** [January 27, 2016, 1:04pm UTC](https://discuss.elastic.co/t/-ttl-problems-bulk-deletion-failures/40222/5 "2016-01-27T13:04:45Z")
The documents are not suppose to be updated as such (log events), But I am parsing the same log file more than one times (some times) and that would update the document version. Thank you for your help but for me the problem is solved an I will not have more time to dig in to the cause. ---
**Author:** ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy)\ **Post date:** [January 27, 2016, 1:07pm UTC](https://discuss.elastic.co/t/-ttl-problems-bulk-deletion-failures/40222/6 "2016-01-27T13:07:04Z")
Ok, cool 🙂 ---
**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\ **Post date:** [January 27, 2016, 2:57pm UTC](https://discuss.elastic.co/t/-ttl-problems-bulk-deletion-failures/40222/7 "2016-01-27T14:57:10Z")
> [@tswider](#): > > I am motivated now to remove the \_ttl and use dated indexes. That's a terrific idea. TTL is deprecated in Elasticsearch 2+ anyway. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:21pm UTC](https://discuss.elastic.co/t/-ttl-problems-bulk-deletion-failures/40222/8 "2017-07-05T23:21:02Z")