I want to create rule where if any change in the user's location is observed i should get an alert for the same. I am using VPN geo location for tracking the same with respect to user.
I am trying o achieve the use case but not getting any luck.
Do you mean any change of location? In other words, if user 1 is usually seen in location A but after a few observations:
A,A,A,A,A,A,A,A,A,A,A,A,A,A,A,B <--- B is the new location
then anomaly detection (using the rare function) is appropriate here.
However, if user 2 is always in a different location:
A,B,C,D,E,F,G,H,I,J,K <------always a different location
Would you want to be alerted 1 time for user 1 and ten times for user 2?
If it is ten times for user 2 then you will not be able to use anomaly detection (because it is always different). You will instead need to maintain a state table for every user's last location.
Thanks Richcolier for the solution.
I am only expecting output in 1st case. I have configured the ML job and it is triggering.
But it is not considering entire months data. Can you suggest which field i should alter for eg:
Query delay: 95985 ms
frequency: 600s
Scroll size: 1000
Or there is any other option to get the desired output.
We tried testing the rule which we created, for eg. I tried login from one location several times and then logged in from another location but the anomaly didn't trigger for the events. Data feed is set to live search from last 1 month start time.
Any specific reason or any misconfiguration which may cause this.
It's likely that you haven't established enough of a baseline. It is not necessarily the "amount" of time, but rather the number of observations of the login events. If you have a month's worth of data, but still only have 5 observations of a login for a particular user (4 from the "normal" place and one from a "non-normal" place) this is not enough. You really need dozens (if not more) observations in order to have the ML model establish the pattern of what's "normal".
How many login events from that user do you actually have before testing the "non-normal" one?
Thanks Richcollier,
My rule start triggering but there is one issue observed that it is not aggregating on one specific field. For ex: If the user is same for multiple anomalies I can see different anomalies for the same user I want to view it in single anomaly and anomaly score for the specific user should increase based on the number of events. How can I achieve that?
consolidating multiple anomalies into a single alert is logic that you'd have to build in the alert using Watcher.
In the Watch, you would look over an interval of time (let's say a week or a month), aggregate anomalies by user, and then apply your own heuristics around scoring that user based on the number of anomalies seen.
If you don't know how to use Watcher with ML jobs, may I recommend Chapter 6
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.