# Is ML capable to detect RAM saturation?

**URL:** <https://discuss.elastic.co/t/is-ml-capable-to-detect-ram-saturation/138185>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-machine-learning\
**Created:** [July 2, 2018, 11:11am UTC](https://discuss.elastic.co/t/is-ml-capable-to-detect-ram-saturation/138185 "2018-07-02T11:11:32Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![genPz01](https://avatars.discourse-cdn.com/v4/letter/g/eada6e/32.png) [@genPz01](https://discuss.elastic.co/u/genPz01)\
**Post date:** [July 2, 2018, 11:11am UTC](https://discuss.elastic.co/t/is-ml-capable-to-detect-ram-saturation/138185/1 "2018-07-02T11:11:32Z")

</div>

Hi,

I test with ML a situation where RAM usage keeps increasing linearly and eats almost all RAM capacity.  
The Machine Learning constructs the appropriate model but doesn't consider the fact that RAM usage is so close to saturation an anomaly.

Is there anything to do to let ML consider this type of case ?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 2, 2018, 11:28am UTC](https://discuss.elastic.co/t/is-ml-capable-to-detect-ram-saturation/138185/2 "2018-07-02T11:28:39Z")

</div>

If RAM usage is always increasing linearly, that is the normal behaviour and it reaching a specific level is not really an anomaly. If you have a fixed limit, that sounds like a use case for Watcher rather than ML.

---

<div class="post-metadata">

**Author:** ![genPz01](https://avatars.discourse-cdn.com/v4/letter/g/eada6e/32.png) [@genPz01](https://discuss.elastic.co/u/genPz01)\
**Post date:** [July 2, 2018, 12:02pm UTC](https://discuss.elastic.co/t/is-ml-capable-to-detect-ram-saturation/138185/3 "2018-07-02T12:02:40Z")

</div>

Do you mean setting a **static** threshold in Watcher (ex: send alert if RAM usage \> 3 go) ?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 2, 2018, 12:11pm UTC](https://discuss.elastic.co/t/is-ml-capable-to-detect-ram-saturation/138185/4 "2018-07-02T12:11:49Z")

</div>

Not sure exactly what the condition would look like, but it does not necessarily need to be static.

---

<div class="post-metadata">

**Author:** ![genPz01](https://avatars.discourse-cdn.com/v4/letter/g/eada6e/32.png) [@genPz01](https://discuss.elastic.co/u/genPz01)\
**Post date:** [July 2, 2018, 12:26pm UTC](https://discuss.elastic.co/t/is-ml-capable-to-detect-ram-saturation/138185/5 "2018-07-02T12:26:23Z")

</div>

Could you please explain a little bit more how ?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 2, 2018, 12:32pm UTC](https://discuss.elastic.co/t/is-ml-capable-to-detect-ram-saturation/138185/6 "2018-07-02T12:32:23Z")

</div>

What does your data look like? Can you describe the condition you are looking to alert on?

---

<div class="post-metadata">

**Author:** ![genPz01](https://avatars.discourse-cdn.com/v4/letter/g/eada6e/32.png) [@genPz01](https://discuss.elastic.co/u/genPz01)\
**Post date:** [July 2, 2018, 12:48pm UTC](https://discuss.elastic.co/t/is-ml-capable-to-detect-ram-saturation/138185/7 "2018-07-02T12:48:10Z")

</div>

I have gathered host metrics from my test machine using MetricBeat.  
I applied ML job on a metric called _system.memory.used.bytes_ (which is the RAM usage in bytes).  
Now as a hypothesis, let's say that the RAM usage keeps growing linearly over time. I want to know if ML will realize that the system RAM has a limit, and therefore consider an anomaly the fact that RAM usage is nearly hits that limit.

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [July 3, 2018, 12:34pm UTC](https://discuss.elastic.co/t/is-ml-capable-to-detect-ram-saturation/138185/8 "2018-07-03T12:34:40Z")

</div>

ML cannot possibly automatically know that your linearly-growing metric has a practical limit. ML only knows the data's behavior in a naive way - meaning, it only knows things about data with respect to observations it has seen in the data's past.

However, to Christian's point - if _you_ know there's an actual limit, then your use-case may be better suited as a static threshold in a Watch rather an ML job.

---

<div class="post-metadata">

**Author:** ![genPz01](https://avatars.discourse-cdn.com/v4/letter/g/eada6e/32.png) [@genPz01](https://discuss.elastic.co/u/genPz01)\
**Post date:** [July 4, 2018, 12:34pm UTC](https://discuss.elastic.co/t/is-ml-capable-to-detect-ram-saturation/138185/9 "2018-07-04T12:34:17Z")

</div>

Clear answer.  
Thank you all for your responses.

---

<div class="post-metadata">

**Author:** ![DavidSztykman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidsztykman/32/24912_2.png) [@DavidSztykman](https://discuss.elastic.co/u/DavidSztykman)\
**Post date:** [July 6, 2018, 6:03am UTC](https://discuss.elastic.co/t/is-ml-capable-to-detect-ram-saturation/138185/10 "2018-07-06T06:03:40Z")

</div>

You could use ML and the max function on system.memory.actual.used.pct which represents the percentage of RAM used in the system.  
If the value increases a lot it’ll trigger an anomaly but technically if the value change from 1% of RAM used to 10% that would generate a pretty high anomaly score even if you still have a lot of RAM available.

It’d probably used watcher for your requirement.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 30, 2018, 10:21pm UTC](https://discuss.elastic.co/t/is-ml-capable-to-detect-ram-saturation/138185/11 "2018-10-30T22:21:37Z")

</div>


