Machine Learning model issue

[Marwan_Ezz]

I'm using Elasticsearch machine learning advanced model to detect anomalies in the trend of our failures.

I'm having an issue that the failure count is increasing everyday at the same time so the model have learnt that it is the normal trend however it is not normal and it should be detected as anomaly.

I tried to use the custom rules in the model but I have multiple services which are having variance in the count so I can't use the actual or typical.

What is the work around in this scenario?

[valeriy42 (Valeriy Khakhutskyy)]

Hello @Marwan_Ezz ,

you need to revert the model to a snapshot before the regular failure count has started in your data.

Then you need to use customer rule e.g. with calendar event and action skip_model for the time period when the failure increase happens.

If what you want the model not to learn doesn't happen at a regular time, it's difficult to say how to formulate the rule in general. You need to provide more information and maybe some single metric viewer graphs.

[richcollier]

@Marwan_Ezz

I'm having an issue that the failure count is increasing everyday at the same time so the model have learnt that it is the normal trend however it is not normal and it should be detected as anomaly.

Not to get too philosophical, but if it changes every day it actually is normal - that's what it does.