# Elaticsearch kibana ml jobs issue **URL:** \ **Category:** Kibana\ **Tags:** elastic-stack-machine-learning\ **Created:** [October 19, 2020, 7:17am UTC](https://discuss.elastic.co/t/elaticsearch-kibana-ml-jobs-issue/252486 "2020-10-19T07:17:25Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Akhil\_Mohan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akhil_mohan/32/77409_2.png) [@Akhil\_Mohan](https://discuss.elastic.co/u/Akhil_Mohan)\ **Post date:** [October 19, 2020, 7:17am UTC](https://discuss.elastic.co/t/elaticsearch-kibana-ml-jobs-issue/252486/1 "2020-10-19T07:17:25Z")
I have created one kibana ml jobs to alert when low event rate and i have ingested data using the audit beat continuously and stop the ingestion but event is not getting triggered.how can i solve this issue. ---
**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\ **Post date:** [October 19, 2020, 2:44pm UTC](https://discuss.elastic.co/t/elaticsearch-kibana-ml-jobs-issue/252486/2 "2020-10-19T14:44:00Z")
Most likely, you didn't allow the "normal" behavior to be learned long enough before triggering your test. More info on what you tried would be helpful here ---
**Author:** ![Akhil\_Mohan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akhil_mohan/32/77409_2.png) [@Akhil\_Mohan](https://discuss.elastic.co/u/Akhil_Mohan)\ **Post date:** [October 20, 2020, 4:54am UTC](https://discuss.elastic.co/t/elaticsearch-kibana-ml-jobs-issue/252486/3 "2020-10-20T04:54:18Z")
basically i have created an anomaly job for event low rate.my aim is to trigger when a low event rate occurs. i have used single metric job and choose low count(Event rate as field) . i created a real time running job and watch for warning (score 0 and above) which is run every 15 minutes.i have indexed the data continuously and data flow is available in graph but the anomaly is not identified and alert is not fired.can you help me to solve the issue. ---
**Author:** ![Akhil\_Mohan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akhil_mohan/32/77409_2.png) [@Akhil\_Mohan](https://discuss.elastic.co/u/Akhil_Mohan)\ **Post date:** [October 20, 2020, 5:59am UTC](https://discuss.elastic.co/t/elaticsearch-kibana-ml-jobs-issue/252486/4 "2020-10-20T05:59:57Z")
![ml-anomoly](https://us1.discourse-cdn.com/elastic/original/3X/9/8/98196529d30a61c7bd946adddd64e31d9a3668c7.png) ---
**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\ **Post date:** [October 20, 2020, 11:02am UTC](https://discuss.elastic.co/t/elaticsearch-kibana-ml-jobs-issue/252486/5 "2020-10-20T11:02:11Z")
What you describe in words is the correct approach, but something must be wrong with your setup here - meaning, whatever index you pointed ML at, or how you're querying it, is not correct. As seen in the screenshot above, the majority of the observations for the count are zero - it is zero almost the entire time (except for the two bumps in the graph). I assume the index that you have ML looking at actually does have data in it? Can you provide the details of your datafeed configuration on the ML job and also perhaps a screenshot of the index that you're monitoring with ML from the Discover tab in Kibana? I'd like to see: ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/1/015969ade85dfdc250ebc5833466c2d731c64ae2.png) and ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/d/ed4ec494537cb2f5d14a59b3db23d312a4d1d9b4.png) Redact any sensitive information as necessary ---
**Author:** ![Akhil\_Mohan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akhil_mohan/32/77409_2.png) [@Akhil\_Mohan](https://discuss.elastic.co/u/Akhil_Mohan)\ **Post date:** [October 20, 2020, 12:07pm UTC](https://discuss.elastic.co/t/elaticsearch-kibana-ml-jobs-issue/252486/6 "2020-10-20T12:07:06Z")
please have a look added screenshot of Discover tab and ml data-feed ![dashboard](https://us1.discourse-cdn.com/elastic/original/3X/6/e/6e81945278d148575d2f37af564b65941bc9153e.png) ![ml-datafeed](https://us1.discourse-cdn.com/elastic/original/3X/8/2/8283be4946445a4f20cb043dc252b8e3babdb661.png) ---
**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\ **Post date:** [October 20, 2020, 1:49pm UTC](https://discuss.elastic.co/t/elaticsearch-kibana-ml-jobs-issue/252486/8 "2020-10-20T13:49:25Z")
I'm a little confused here. Previous screenshots were for a job name `ml-latest-job`, but this is for a job named `ml-job-auditbeat`. Can you please provide consistent information? It would make things easier to help you. Assuming these jobs are similar, however, and if you really are looking for `low_count` of events in the index `auditbeat-*` and if that index really does have data flowing into it in a timely manner (that doesn't have excessive ingest delays) then you should be just fine. However, clearly something in your setup is incorrect, so we need to find it. First, to rule out a real-time ingest delay issue, let's walk through the config of a new job and see what you get along the way Step 1) Create a new ML job, pick `auditbeat-*` as the index, and choose Single Metric as the job type. Step 2) In the following screen, click the "Use full auditbeat-\* data" button. It should look something like this: ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/a/6acdb7a806f9dba415d4c6ec5326dbe6584f5c17.png) If it DOES NOT look something like the above (i.e. has a set of bars representing index volume over time similar to what you'd see in Kibana Discover), then stop here and let us know. Step 3) If it does look similar to the above, then feel free to continue the configuration (by pressing the Next button) and picking `Low count (Event rate)` from the dropdown ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/0/50e3ee0e2f3eedb10f7445d25cdbd140ef6204f7.png) Step 4) Click Next, and name the job: ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/2/8280c0a6417ca3691f1c21569c99da3df58c003f.png) Step 5) Click the Next button twice and then the "Create Job" button. You should see ML run and point out potential anomalies in the data: ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/d/ed615b30ebca4adeed0a26fa5b0f974727d4b954.png) (you can see in the example, that the period of lower volume was indeed flagged as anomalous) Step 6) Optionally, click "Start job running in real-time" Let me know which of the above steps do not work for you. ---
**Author:** ![Akhil\_Mohan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akhil_mohan/32/77409_2.png) [@Akhil\_Mohan](https://discuss.elastic.co/u/Akhil_Mohan)\ **Post date:** [October 20, 2020, 2:28pm UTC](https://discuss.elastic.co/t/elaticsearch-kibana-ml-jobs-issue/252486/9 "2020-10-20T14:28:29Z")
Hi i have created new ml job (auditbeat-test-ml) for low count and which is pointing to auditbeat-\* step1 ![screenshot-localhost_5601-2020.10.20-19_40_11](https://us1.discourse-cdn.com/elastic/original/3X/c/c/ccf7211ba9edd215667b2ea83d293209c3340b24.png) step2 ![screenshot-localhost_5601-2020.10.20-19_40_49](https://us1.discourse-cdn.com/elastic/original/3X/e/6/e61562929f6dd96aba5482d91c4644b732a90049.png) step3 ![screenshot-localhost_5601-2020.10.20-19_41_38](https://us1.discourse-cdn.com/elastic/original/3X/c/7/c70eacea9a0752f5b06aa2f31951d0e900320226.png) step4 ![screenshot-localhost_5601-2020.10.20-19_42_53](https://us1.discourse-cdn.com/elastic/original/3X/0/6/06f5c864174b03aae01f8dac3565c2516dbc1063.png) Finally created watcher for anomaly score for minor( 25 and greater). data feed of newly created job ![screenshot-localhost_5601-2020.10.20-19_47_48](https://us1.discourse-cdn.com/elastic/original/3X/e/9/e9d5961e896d174e50ca43596139c100466082c5.png) ---
**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\ **Post date:** [October 20, 2020, 2:49pm UTC](https://discuss.elastic.co/t/elaticsearch-kibana-ml-jobs-issue/252486/10 "2020-10-20T14:49:00Z")
Great! Looks like that worked. Now, keep in mind that you only have less than 24 hours worth of data in that auditbeat index (earliest data seems to be today, at 17:00 hours in your time zone). Now - let this job run for a few days. ---
**Author:** ![Akhil\_Mohan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akhil_mohan/32/77409_2.png) [@Akhil\_Mohan](https://discuss.elastic.co/u/Akhil_Mohan)\ **Post date:** [October 20, 2020, 3:12pm UTC](https://discuss.elastic.co/t/elaticsearch-kibana-ml-jobs-issue/252486/11 "2020-10-20T15:12:18Z")
hai Do we have any proper documentation toadd this ml job in siem detection can you give spme instruction to add ml job in siem detection. my aim is to trigger alert from siem detection when low event rate anomoly occurs ---
**Author:** ![richcollier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/richcollier/32/115035_2.png) [@richcollier](https://discuss.elastic.co/u/richcollier)\ **Post date:** [October 20, 2020, 3:34pm UTC](https://discuss.elastic.co/t/elaticsearch-kibana-ml-jobs-issue/252486/12 "2020-10-20T15:34:32Z")
> **[Creating detection rules | Elastic Security Solution \[7.9\] | Elastic](https://www.elastic.co/guide/en/security/current/rules-ui-create.html)** ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 17, 2020, 3:34pm UTC](https://discuss.elastic.co/t/elaticsearch-kibana-ml-jobs-issue/252486/13 "2020-11-17T15:34:35Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.