Cant' import Machine Learning Anomaly Jobs - I have Enterprise Cloud Subscription

Elastic Stack Kibana
1

Hi,
We are an Elastic Cloud Enterprise customer. The machine learning jobs that are to be prebuilt are missing.

ML-screenshot
ML-screenshot1527×655 43.5 KB

I have a ML node assigned to the cluster, i can't find where to import them.

Cluster is running 7.10.2

Thank you

2

Hi @opiedrah

So if this is a brand new cluster there will be no jobs until

a) You ingest data via an integration or a beat they will be created during the setup process example Metrics, APM etc...

b) You manually create one

Which Job were you expecting to see?

1 Like
3

There are indeed some "pre-built" jobs for the SIEM app and APM, but you need to enable/import those jobs from those apps - not from the general ML UI.

For example, reference: https://www.elastic.co/guide/en/siem/guide/current/machine-learning.html

4

When i click under SIEM, ML job Settings it shows everything failed, and when i click on each one it shows no job.

ml-job-settings
ml-job-settings864×797 88.8 KB

No Job

NoJob
NoJob1251×655 34.9 KB

5

The error message indicates that there is no data in your cluster that these ML jobs require in order to install and run. Refer to the job definitions which show the required data necessary: https://www.elastic.co/guide/en/siem/guide/current/prebuilt-ml-jobs.html

For example, you can see that the linux_anomalous_network_url_activity_ecs requires data collected by from Auditbeat (Linux). And, as you can imagine, the ML jobs whose names begin with windows requires data collected via winlogbeat.

Unless you've collected that kind of data using those beats, the product is working as expected by not allowing you to enable jobs on data that doesn't yet exist.

6

Interesting... We are currently using fleet with the endpoint security integration, this captures security events , network, process etc. So what you are saying is that the ML jobs currently do not work with data from fleet+ElasticEndpoint. One would also have to install winlogbeat for the ML jobs to work?

Thanks

7

Thank you for the additional information. Yes, there was a compatibility issue with built-in ML jobs not recognizing the indices if the data was collected via the new, consolidated Elastic Agent (since the names of the indices are different than if collected with legacy Beats). Word is that this is fixed in v7.11 due out next week.

8

Thank you @richcollier I'll mark it as resolved and hope for the best for 7.11 or 7.12. Do you have any insights on this, or know someone that does?

Thanks again

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.