# ML : detects unusually low number of users

**URL:** https://discuss.elastic.co/t/ml-detects-unusually-low-number-of-users/346057
**Category:** Kibana
**Tags:** elastic-stack-machine-learning
**Created:** [October 30, 2023, 3:54pm UTC](https://discuss.elastic.co/t/ml-detects-unusually-low-number-of-users/346057 "2023-10-30T15:54:38Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![AmS](https://avatars.discourse-cdn.com/v4/letter/a/a4c791/32.png) [@AmS](https://discuss.elastic.co/u/AmS)
#### Post date: [October 30, 2023, 3:54pm UTC](https://discuss.elastic.co/t/ml-detects-unusually-low-number-of-users/346057/1 "2023-10-30T15:54:39Z")

</div>

Hello ,  
I'm using machine learning detector on version 7.17.  
I would like to detect unusually low number of users.  
I m using for that low\_distinct\_count as function.  
It works fine when at least there is at least one user .  
Exemple

- if the number of user goes from 17 to 1 , this decrease is well detected
- if the number of user goes from 17 to 0 , this decrease is NOT detected

Are there any condition to add to the ML job so we can detect event if all users are lost?

Thanks

---

<div class="post-metadata">

### Author: ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)
#### Post date: [October 30, 2023, 5:30pm UTC](https://discuss.elastic.co/t/ml-detects-unusually-low-number-of-users/346057/2 "2023-10-30T17:30:37Z")

</div>

This, in fact, should work as you are expecting it to. I'd love to see evidence (like a screenshot) of the situation that you describe with it not working! Please post here!

---

<div class="post-metadata">

### Author: ![AmS](https://avatars.discourse-cdn.com/v4/letter/a/a4c791/32.png) [@AmS](https://discuss.elastic.co/u/AmS)
#### Post date: [October 30, 2023, 10:01pm UTC](https://discuss.elastic.co/t/ml-detects-unusually-low-number-of-users/346057/3 "2023-10-30T22:01:12Z")

</div>

Here the two cases

- it works when at least one user is present  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/d/cd64b6ce10db1cde84a6137f74250147fa04a68e.png)  
-it does not work there is no users  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/e/ce029de04b8e687aab9fe73f31df5bb26f16afcb.png)  
as you can see in the second graph detection of the decrease starts on 28/10 rather then the 26/10 like the first one.

FYI , I'm using a detector like " low\_distinct\_count(ID) by XYZ "

---

<div class="post-metadata">

### Author: ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)
#### Post date: [November 2, 2023, 6:05pm UTC](https://discuss.elastic.co/t/ml-detects-unusually-low-number-of-users/346057/4 "2023-11-02T18:05:54Z")

</div>

I stand corrected - it does ignore empty buckets, which is a little counter-intuitive to me but apparently, that's how it was designed. I've asked dev to consider making a feature enhancement to make the behavior optional (like we do by having `count` and `non_zero_count` function variants).

In the meantime, this can be accomplished via a workaround.

1. Use aggregations in the datafeed to calculate the cardinality of your field of choice. See examples here: [Aggregating data for faster performance | Machine Learning in the Elastic Stack [8.10] | Elastic](https://www.elastic.co/guide/en/machine-learning/current/ml-configuring-aggregation.html)
2. Use the `low_sum` detector function on the aggregated field name.

---

<div class="post-metadata">

### Author: ![AmS](https://avatars.discourse-cdn.com/v4/letter/a/a4c791/32.png) [@AmS](https://discuss.elastic.co/u/AmS)
#### Post date: [November 3, 2023, 11:18am UTC](https://discuss.elastic.co/t/ml-detects-unusually-low-number-of-users/346057/5 "2023-11-03T11:18:09Z")

</div>

Ok ,  
Low\_sum is not adequate since that the ID is not agregable field.  
I will try the agregation on the datafeed

---

<div class="post-metadata">

### Author: ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)
#### Post date: [November 10, 2023, 6:49pm UTC](https://discuss.elastic.co/t/ml-detects-unusually-low-number-of-users/346057/6 "2023-11-10T18:49:27Z")

</div>

I got it to work via the cardinatlity agg and the low\_sum function:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/1/21d0872d8334cd72ed5017ba27b839ab1f080af6.jpeg)

job config:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/2/92bcb0741c83a951dc4bb027cad960675af3ba69.jpeg)

Note the name of the cardinality agg (here it is `dc_airline`) is the same as what's used in the detector definition (`low_sum(dc_airline)`) and the value of the `summary_count_field_name`

---

<div class="post-metadata">

### Author: ![AmS](https://avatars.discourse-cdn.com/v4/letter/a/a4c791/32.png) [@AmS](https://discuss.elastic.co/u/AmS)
#### Post date: [November 20, 2023, 8:39am UTC](https://discuss.elastic.co/t/ml-detects-unusually-low-number-of-users/346057/7 "2023-11-20T08:39:38Z")

</div>

Thanks for your answer ,  
I didn't succeed to have the expected result.  
I have done the same thing , But I need to add a field XYZ as influencer:  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/9/19f6dee6841afdae35cae1932253e2b4fd343e12.png)

in the result of the job I Don't have XYZ :

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/1/f1939abf8ba883b261384de49d5b64d701b44d24.png)

---

<div class="post-metadata">

### Author: ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)
#### Post date: [November 20, 2023, 1:55pm UTC](https://discuss.elastic.co/t/ml-detects-unusually-low-number-of-users/346057/8 "2023-11-20T13:55:02Z")

</div>

If you intend to split by using a by\_field or a partition\_field then your datafeed query has to also include a `terms` aggregation so that you get a `service_cardinality` value for every `XYZ`.

I suspect you don't have that at the moment

See example

> **[Aggregating data for faster performance | Machine Learning in the Elastic...](https://www.elastic.co/guide/en/machine-learning/current/ml-configuring-aggregation.html#aggs-using-date-histogram)**

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 18, 2023, 1:55pm UTC](https://discuss.elastic.co/t/ml-detects-unusually-low-number-of-users/346057/9 "2023-12-18T13:55:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
