# Watcher for specific field in ML Job **URL:** \ **Category:** Kibana\ **Tags:** elastic-stack-machine-learning, elastic-stack-alerting\ **Created:** [May 9, 2020, 4:22pm UTC](https://discuss.elastic.co/t/watcher-for-specific-field-in-ml-job/231852 "2020-05-09T16:22:32Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![vdelcampo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vdelcampo/32/65731_2.png) [@vdelcampo](https://discuss.elastic.co/u/vdelcampo)\ **Post date:** [May 9, 2020, 4:22pm UTC](https://discuss.elastic.co/t/watcher-for-specific-field-in-ml-job/231852/1 "2020-05-09T16:22:32Z")
Hi. Im monitoring all traffic that comes from my Big Ip F5 Load Balancer. I have an index with all this information and a ml job with a count function and a detector using two fields (virtual\_ip.keyword over http\_path.keyword). Now I want to create a watcher for a specific http\_path value. Its possible? Thanks in advance ---
**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\ **Post date:** [May 10, 2020, 10:58am UTC](https://discuss.elastic.co/t/watcher-for-specific-field-in-ml-job/231852/2 "2020-05-10T10:58:06Z")
Hi - yes, of course you can create a Watch for any ML job - it only requires a little knowledge of how watcher works and how to get the information about the anomalies out of either the [ML results API](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/ml-apis.html#ml-api-result-endpoint) or via querying of the `.ml-anomalies-*` indices. If you're not familiar with how Watcher works with ML you can first take a look [at this blog](https://www.elastic.co/blog/alerting-on-machine-learning-jobs-in-elasticsearch-v55). Although it is a little outdated, the fundamentals are still relevant. Additionally, it is also good to understand how anomalies are scored and their different flavors (bucket, record, influencer). For info on that, please [consult this blog](https://www.elastic.co/blog/machine-learning-anomaly-scoring-elasticsearch-how-it-works). I don't quite understand your specific use case and what you're looking for. Perhaps you can more succinctly describe what you want to accomplish and what your current ML job config is. ---
**Author:** ![vdelcampo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vdelcampo/32/65731_2.png) [@vdelcampo](https://discuss.elastic.co/u/vdelcampo)\ **Post date:** [May 10, 2020, 12:29pm UTC](https://discuss.elastic.co/t/watcher-for-specific-field-in-ml-job/231852/3 "2020-05-10T12:29:56Z")
Hi Rich. I want to create a watcher that sends an email if "http\_path": "my\_path/mypath.aspx" has less than X requests. For example, based on my ml job results, if "my\_path/mypath.aspx" has 2500 requests per minute from 9:00 am to 17:30 pm and at 15:00 this value is minor, the watcher must send an email. Thanks a lot. Kind Regards Víctor ---
**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\ **Post date:** [May 10, 2020, 1:48pm UTC](https://discuss.elastic.co/t/watcher-for-specific-field-in-ml-job/231852/4 "2020-05-10T13:48:04Z")
Hi Victor - a couple of considerations here: 1. Be careful describing the use case using "over" as that implies that you are using Population Analysis instead of Temporal ([https://www.elastic.co/blog/temporal-vs-population-analysis-in-elastic-machine-learning](https://www.elastic.co/blog/temporal-vs-population-analysis-in-elastic-machine-learning)). In Population Analysis you'd be comparing the count of every `http_path` against other `http_path`s . In Termporal, you'd be comparing a particular count of each and every `http_path` against its own history. 2. If you want to do Temporal, you need to be careful of the cardinality of the `http_path` field. If it is in the tens of thousands or more, you may need to adjust the `model_memory_limit`. ---
**Author:** ![vdelcampo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vdelcampo/32/65731_2.png) [@vdelcampo](https://discuss.elastic.co/u/vdelcampo)\ **Post date:** [May 10, 2020, 5:08pm UTC](https://discuss.elastic.co/t/watcher-for-specific-field-in-ml-job/231852/5 "2020-05-10T17:08:17Z")
Hi. When i was modeling the ml job, i thought using Population Analysis was the better way to do what i wanted. If i use a temporal Analysis using http\_path, the cardinality was huge, but if i use population using virtual\_ip over http\_path, i got all http\_path (more than 100) group by virtual\_ip (only 5). I read on that link its possible to filter the Datafeed, so maybe this way is good to apply a watcher only for the http\_path value i want to monitor. What do you think? Regards Víctor ---
**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\ **Post date:** [May 10, 2020, 5:47pm UTC](https://discuss.elastic.co/t/watcher-for-specific-field-in-ml-job/231852/6 "2020-05-10T17:47:40Z")
Population Analysis solves the cardinality problem, but it isn't the right approach when the "population" isn't homogeneous. So, if there are some http\_paths that are much more popular then others (i.e. they routinely get more hits) - then, relative to their peers, those will always be labeled as unusual, despite consistent behavior with respect to themselves. In temporal analysis, if you don't want to include every single http\_path in the analysis, you could [filter the datafeed](https://www.elastic.co/blog/filtering-input-data-to-refine-machine-learning-jobs), or you could use a [custom aggregation](https://www.elastic.co/guide/en/machine-learning/7.6/ml-configuring-aggregation.html) (like a [terms aggregation](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html) with a `size` option ) to only return the "Top N" most popular http\_paths and analyze those. ---
**Author:** ![vdelcampo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vdelcampo/32/65731_2.png) [@vdelcampo](https://discuss.elastic.co/u/vdelcampo)\ **Post date:** [May 11, 2020, 8:44am UTC](https://discuss.elastic.co/t/watcher-for-specific-field-in-ml-job/231852/7 "2020-05-11T08:44:23Z")
That´s perfect. I filtered the datafeed and now its easy to setup a watch only for the http\_path i wanted. Thanks a lot! Regards Víctor ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 8, 2020, 8:44am UTC](https://discuss.elastic.co/t/watcher-for-specific-field-in-ml-job/231852/8 "2020-06-08T08:44:25Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.