# Can't delete ML jobs which was on hard\_limit

**URL:** <https://discuss.elastic.co/t/cant-delete-ml-jobs-which-was-on-hard-limit/202634>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-machine-learning\
**Created:** [October 8, 2019, 8:47am UTC](https://discuss.elastic.co/t/cant-delete-ml-jobs-which-was-on-hard-limit/202634 "2019-10-08T08:47:17Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![EunBae\_Park](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eunbae_park/32/55523_2.png) [@EunBae\_Park](https://discuss.elastic.co/u/EunBae_Park)\
**Post date:** [October 8, 2019, 8:47am UTC](https://discuss.elastic.co/t/cant-delete-ml-jobs-which-was-on-hard-limit/202634/1 "2019-10-08T08:47:17Z")

</div>

Hi, This is Jason.

I'm in trouble for deleting hard-limited ml-job.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/4/94a53baff44b20393224682a3ef34fe6b1d48b31.png)

I'd deleted them yesterday, however It's still deleting ☹  
I used dev-tool to delete them again today, but failed like below

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/d/8d8174733d872306cb34451bd3cd3693a1d4f996.png)

How can I delete these jobs?

Thanks & regards  
Jason

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [October 8, 2019, 2:08pm UTC](https://discuss.elastic.co/t/cant-delete-ml-jobs-which-was-on-hard-limit/202634/2 "2019-10-08T14:08:42Z")

</div>

I suspect that because these jobs were "too big" for your setup (hence giving you a `hard_limit`) then the deleting of these jobs (which also has to remove docs from some of the `.ml-*` indices) is also very taxing on your system.

First, some obvious questions:

1. What version are you on?
2. How many jobs have you configured?
3. How many are actively running?
4. If you were to invoke the following API call in DevTools console several times:

`GET _cat/indices/.ml-*?v`

and look at the `docs.count` and `docs.deleted` columns, do you notice if those numbers are changing significantly? In other words, is there an active background task that is still trying to delete stuff?

---

<div class="post-metadata">

**Author:** ![EunBae\_Park](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eunbae_park/32/55523_2.png) [@EunBae\_Park](https://discuss.elastic.co/u/EunBae_Park)\
**Post date:** [October 10, 2019, 12:09am UTC](https://discuss.elastic.co/t/cant-delete-ml-jobs-which-was-on-hard-limit/202634/3 "2019-10-10T00:09:16Z")

</div>

1. What version are you on?  
**- v6.8.0**
2. How many jobs have you configured?  
**- 27 jobs total, 26 running jobs and 25 datafeeds**
3. How many are actively running?  
**- 26 jobs running**
4. If you were to invoke the following API call in DevTools console several times:  
**- yes about 3-5 times**

I saw this morning the deprecated jobs have been deleted completly. It took 4 days for deleteing, wow.  
I think they were too big.

Thanks fot the tip

> `GET _cat/indices/.ml-*?v`
> 
> and look at the `docs.count` and `docs.deleted` columns, do you notice if those \>numbers are changing significantly? In other words, is there an active background task \>that is still trying to delete stuff?

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [October 10, 2019, 12:59am UTC](https://discuss.elastic.co/t/cant-delete-ml-jobs-which-was-on-hard-limit/202634/4 "2019-10-10T00:59:22Z")

</div>

Since your are on v6.8.0 you might also want to consider the following:

```auto
DELETE _ml/_delete_expired_data

```

This request normally occurs every night with our nightly maintenance, but [there is a bug](https://github.com/elastic/elasticsearch/issues/47003) in our nightly maintenance scheduler. The bug is fixed in 6.8.4 and 7.4.1. But, since this bug exists in your version, your jobs will continually accumulate results and state without bound.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 7, 2019, 1:07am UTC](https://discuss.elastic.co/t/cant-delete-ml-jobs-which-was-on-hard-limit/202634/5 "2019-11-07T01:07:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.