# Job soft limit

**URL:** <https://discuss.elastic.co/t/job-soft-limit/187182>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-machine-learning\
**Created:** [June 24, 2019, 5:51pm UTC](https://discuss.elastic.co/t/job-soft-limit/187182 "2019-06-24T17:51:52Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bharath\_Kumar\_R](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bharath_kumar_r/32/45474_2.png) [@Bharath\_Kumar\_R](https://discuss.elastic.co/u/Bharath_Kumar_R)\
**Post date:** [June 24, 2019, 5:51pm UTC](https://discuss.elastic.co/t/job-soft-limit/187182/1 "2019-06-24T17:51:52Z")

</div>

While the job I created was running, suddenly its memory status changed to soft\_limit. Can you explain the cause and remedy for this situation? My Elasticsearch is running on an ec2 instance(can this be a reason for this situation ?)

 ![26%20PM](https://us1.discourse-cdn.com/elastic/original/3X/8/f/8f13d3cc385cc0a944ea37fde5153087bc48195d.png)

the below is the job description:  
{  
"count" : 1,  
"jobs" : [  
{  
"job\_id" : "my\_job\_low\_sum",  
"job\_type" : "anomaly\_detector",  
"job\_version" : "7.0.1",  
"create\_time" : 1561121146128,  
"analysis\_config" : {  
"bucket\_span" : "15m",  
"detectors" : [  
{  
"detector\_description" : "sum per method\_status",  
"function" : "low\_sum",  
"field\_name" : "EVENTHOUR",  
"partition\_field\_name" : "SHIPPERID\_CARRIERID",  
"detector\_index" : 0  
}  
],  
"influencers" : [  
"SHIPPERID",  
"CARRIERID"  
]  
},  
"analysis\_limits" : {  
"model\_memory\_limit" : "1024mb",  
"categorization\_examples\_limit" : 4  
},  
"data\_description" : {  
"time\_field" : "EVENTTIME",  
"time\_format" : "epoch\_ms"  
},  
"model\_snapshot\_retention\_days" : 10,  
"model\_snapshot\_id" : "1561390307",  
"results\_index\_name" : "custom-low\_sum\_results"  
}  
]  
}

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [June 25, 2019, 1:03pm UTC](https://discuss.elastic.co/t/job-soft-limit/187182/2 "2019-06-25T13:03:58Z")

</div>

`soft_limit` means that you are getting close to having the job exceed the stated `model_memory_limit` because the number of unique partitions (in your case `SHIPPERID_CARRIERID`) is getting higher and higher.

If you know that the combinations of `SHIPPERID_CARRIERID` is going to be fixed (i.e. not a never-ending number of combinations), then you can appropriately increase the setting of `model_memory_limit` on the job.

A good rule of thumb is to allocate about 30kB per instance of the `partition_field_name`. So, if you have 50,000 possible combinations of `SHIPPERID_CARRIERID`, then set the `model_memory_limit` to `1500mb`.

Again, if you find that it is possible that there is an unlimited number of possible combinations of `SHIPPERID_CARRIERID`, then you are going to have a problem with your approach.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 23, 2019, 1:04pm UTC](https://discuss.elastic.co/t/job-soft-limit/187182/3 "2019-07-23T13:04:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.