# Machine learning anomaly correlation **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-machine-learning\ **Created:** [August 6, 2019, 7:45am UTC](https://discuss.elastic.co/t/machine-learning-anomaly-correlation/193954 "2019-08-06T07:45:45Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![liorg2](https://avatars.discourse-cdn.com/v4/letter/l/ed8c4c/32.png) [@liorg2](https://discuss.elastic.co/u/liorg2)\ **Post date:** [August 6, 2019, 7:45am UTC](https://discuss.elastic.co/t/machine-learning-anomaly-correlation/193954/1 "2019-08-06T07:45:46Z")
0 Is there a way to achieve the following with elastic machine learning: A sample demo structure: Index data has the following fields: job\_duration\_time,server, boot\_time, run\_time the first field:job\_duration\_time, is a summary of the last two: job\_duration\_time=boot\_time+run\_time i would like to achieve: 1. find anomalies in job\_duration\_time by server (i know how to implement: multi metric job checking median of job\_duration\_time splitted to server) 2. **find the root cause**. meaning: find which of of the boot\_time/run\_time has correlation to the first anomaly. example for such correlation: ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/1/01540dee0331e1875b96fde611bda3584b059312.png) ---
**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\ **Post date:** [August 6, 2019, 11:20am UTC](https://discuss.elastic.co/t/machine-learning-anomaly-correlation/193954/2 "2019-08-06T11:20:37Z")
Why not track all 3 metrics, per server in the job? ---
**Author:** ![liorg2](https://avatars.discourse-cdn.com/v4/letter/l/ed8c4c/32.png) [@liorg2](https://discuss.elastic.co/u/liorg2)\ **Post date:** [August 6, 2019, 8:07pm UTC](https://discuss.elastic.co/t/machine-learning-anomaly-correlation/193954/3 "2019-08-06T20:07:38Z")
thanks for the reply @richcollier If each data point represents a minute for example, theoretically- the anomaly of the first mertic, can be 1 minute after the anomaly of the second metric. I'm not a statistician, but I think that you should use a formula to find correlation.(e.g.pearson correlation) ---
**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\ **Post date:** [August 7, 2019, 1:32pm UTC](https://discuss.elastic.co/t/machine-learning-anomaly-correlation/193954/4 "2019-08-07T13:32:41Z")
Pearson Correlation tells you how related (in a linear sense) two variables are on average. You need many observations (irrespective of time). This doesn't make sense in the context of time-series based data where "correlation" really means that something co-occurs in time. Keep in mind that we bucket the data in time (hence the meaningfulness of the [bucket\_span parameter](https://www.elastic.co/blog/explaining-the-bucket-span-in-machine-learning-for-elasticsearch)). In your case, you have 3 metrics where the 3rd is the sum of the first two, so naturally, you will get time-correlation (I disagree with your assertion that there is a 1 sample delay of anomalousness). Looking at the anomaly scores of the 3 time-series will allow you to infer causality. In other words, if metric3 = metric1 + metric2 then when metric3 is odd it is very likely that either metric1 and/or metric2 will also be odd. The scores of metric1 and metric2 are a proxy for which is the most responsible. ---
**Author:** ![liorg2](https://avatars.discourse-cdn.com/v4/letter/l/ed8c4c/32.png) [@liorg2](https://discuss.elastic.co/u/liorg2)\ **Post date:** [August 7, 2019, 1:35pm UTC](https://discuss.elastic.co/t/machine-learning-anomaly-correlation/193954/5 "2019-08-07T13:35:36Z")
thanks @richcollier, i will try and let you know 🙂 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 4, 2019, 1:35pm UTC](https://discuss.elastic.co/t/machine-learning-anomaly-correlation/193954/6 "2019-09-04T13:35:37Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.