ML jobs


i have created the built-in machine learning jobs for windows and linux,
for linux it s working as expected, for windows some of the jobs are showing 0 Processed data.
one of the jobs is "windows_anomalous_process_creation"

i ran the below :
GET /_ml/datafeeds/datafeed-windows_anomalous_process_creation/_preview
and the output is

"@timestamp" : 1580212136848,
"" : "DESKTOP-TEST123",
"" : "GoogleUpdate.exe",
"" : "svchost.exe",
"" : "SYSTEM"
"@timestamp" : 1580213339330,
"" : "DESKTOP-TEST123",
"" : "UsoClient.exe",
"" : "svchost.exe",
"" : "SYSTEM"

Since the datafeed preview looks okay, it seems like the problem is probably with the fact that your windows logs might be ingested more slowly than your linux logs, thus the real-time nature of the ML job is undermined.

You could test this theory by doing the following:

  1. Go to the ML Anomaly Detection Jobs page.
  2. Clone the windows_anomalous_process_creation job
  3. Have the job run on some past data.
  4. Continue to run the data in "real-time"
  5. If the number of processed records doesn't increase after entering real-time mode (and assuming you are still ingesting new Windows logs) - then you know your ingest delay is bigger than what the ML job is accounting for - so you will need to increase the query_delay parameter of the ML job (or figure out why the ingest delay is larger on Windows than Linux)

thank you, it solved the issue

1 Like