Category examples not available in Machine Learning module 5.5

[gautamv (Gautam)]

Hi,

I'm trying out the new Machine learning module 5.5. I'm running a job where I want to identify anomalies using function count and by_field_name mlcategory (categorization_field_name is message).

However, in the anomaly explorer, I only get the mlcategory number, which caused the anomaly. I'm no longer getting category examples of the specific mlcategory. I used to get this in the beta function.

Without this, I'm not able to understand which type of message caused the anomaly.

Please help.

[dmitri (Dimitris Athanasiou)]

Hi,

Have you upgraded that cluster from 5.4 to 5.5? Also, which 5.5.x exactly?

In addition, what do you see if you do a GET at /_xpack/ml/anomaly_detectors/job_id/results/categories (see get categories API)

[vsmirnov (Val)]

I have the same problem (5.5.0). The API GET call to /_xpack/ml/anomaly_detectors/job_id/results/categories returns examples, but ML GUI does not show them - there's no "categories examples" column for some reason.

I noticed that this happens with the job that has detector of type "rare by mlcategory over some_field". For other jobs with "rare by mlcategory partition_field_name=somefield" detector type this doesn't happen.

Is that something known?

[sophie_chang]

Category examples are conditionally displayed, depending on the type of analysis in use.

If a population analysis has been performed (e.g. "rare by mlcategory over process_name" - using the over_field_name), then the category examples are not displayed. In this case, multiple categories may have been considered rare for this process_name. By expanding the row, you can see which categories were considered rare. In the current version, we did not write out these category examples because there could have been up to 10 different categories and up to 4 examples for each.

In order to identify the category example, then use the following in Dev Tools:

GET _xpack/ml/anomaly_detectors/job_id/results/categories/category_id

We understand this is not an ideal extra step, and we will be looking to improve on this in the future.

If the analysis is an individual analysis (i.e. over_field_name is not used) then the category examples should be displayed. In general, if you can see the "mlcategory" value in the "found for" column, then the examples should be displayed.