# Xpack anomaly detect theory

**URL:** <https://discuss.elastic.co/t/xpack-anomaly-detect-theory/138995>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-machine-learning\
**Created:** [July 7, 2018, 2:16am UTC](https://discuss.elastic.co/t/xpack-anomaly-detect-theory/138995 "2018-07-07T02:16:51Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Eric\_Yin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_yin/32/33098_2.png) [@Eric\_Yin](https://discuss.elastic.co/u/Eric_Yin)\
**Post date:** [July 7, 2018, 2:16am UTC](https://discuss.elastic.co/t/xpack-anomaly-detect-theory/138995/1 "2018-07-07T02:16:51Z")

</div>

Dear X-Pack developers,

I have already checked the open source code of Xpack for personal interest, I am not sure which kinds of algorithm is used for machine learning and I also found move average aggregation algorithm in core ES code but did not find any provoke from XPack.

Could you please kindly advice where I can find the algorithm in XPACK? Thank you very much.  
I would be great if you can share some theory of Xpack anomaly detection.

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [July 8, 2018, 10:03am UTC](https://discuss.elastic.co/t/xpack-anomaly-detect-theory/138995/2 "2018-07-08T10:03:35Z")

</div>

Some videos that describe the math behind X-Pack Machine Learning:

2017 ElasticON:  
[https://www.elastic.co/elasticon/conf/2017/sf/machine-learning-and-statistical-methods-for-time-series-analysis](https://www.elastic.co/elasticon/conf/2017/sf/machine-learning-and-statistical-methods-for-time-series-analysis)

2018 ElasticON:  
[https://www.elastic.co/elasticon/conf/2018/sf/the-math-behind-elastic-machine-learning](https://www.elastic.co/elasticon/conf/2018/sf/the-math-behind-elastic-machine-learning)

---

<div class="post-metadata">

**Author:** ![Eric\_Yin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_yin/32/33098_2.png) [@Eric\_Yin](https://discuss.elastic.co/u/Eric_Yin)\
**Post date:** [July 10, 2018, 2:47am UTC](https://discuss.elastic.co/t/xpack-anomaly-detect-theory/138995/3 "2018-07-10T02:47:42Z")

</div>

thanks for your reply, I found the ml module is packed into C++ and watch the video, it looks like the anomaly score and forecast are used by different algorithm.

TScores::compute(jointProbabilityWeight, extremeProbabilityWeight,  
minExtremeSamples, maxExtremeSamples, maximumAnomalousProbability,  
p1, overallScore, overallProbability);

I am still not very clear what kind of algorithm is used to calculate the score and the probability. Is there any guide combined with the code analysis? or some docker image for that ML C++ source code environment which is used to debug more easily?

---

<div class="post-metadata">

**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\
**Post date:** [July 10, 2018, 10:31am UTC](https://discuss.elastic.co/t/xpack-anomaly-detect-theory/138995/4 "2018-07-10T10:31:47Z")

</div>

No, there is no code guide.

The probability calculation comes from measuring the observed sample against the probability model that's been constructed behind the scenes by X-Pack ML. The basis of the approach has been described in this academic paper: [http://www.ijmlc.org/papers/398-LC018.pdf](http://www.ijmlc.org/papers/398-LC018.pdf)

The normalized anomaly scoring approach is described in a practical sense here: [https://www.elastic.co/blog/machine-learning-anomaly-scoring-elasticsearch-how-it-works](https://www.elastic.co/blog/machine-learning-anomaly-scoring-elasticsearch-how-it-works)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 7, 2018, 10:31am UTC](https://discuss.elastic.co/t/xpack-anomaly-detect-theory/138995/5 "2018-08-07T10:31:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
