Dedicated ml index for apm high_mean_transaction_duration

[willemdh]

Hello,

Currently the auto-created ml job apm-pr-d40c-high_mean_transaction_duration saves it's model in the shared ml index. This is problematic, as this index contains sensitive information which we cannot open for developers.

So is it possible to migrate this to a dedicated ml index and use index level privileges? Or do we have to provide granular field-level access controls on job_id in .ml-anomalies-shared?

Also, although the job has been created and run over a period of +- last 60 days, APM does not show any detected anomalies:

When I click on View job, it goes to the configured job:

I was expecting similar integration as in Uptime, where anomalies are shown as red vertical annotation lines? 27/10, there was an anomaly which was higher then 75.

Grtz

Willem

[wei.wang]

Hi Willem,

Two things are being discussed here.

1. The permission about ml jobs result.

We currently are working on Integration with Kibana Spaces, with that being implemented (now is planed for 7.11 release), you will be able to use Kibana Spaces/Indicies feature to solve your problem. Unfortunately at the moment we don’t have a good way to restrict certain users to only view the results of certain jobs.

2. APM does not show any detected anomalies.

APM channel might be a good place to ask around. I also did a quick search in Github, this might be what you are looking for: https://github.com/elastic/kibana/issues/52250 . As it is marked as enhancement, we might haven't decided when to implement it yet.

Cheers and stay safe.

Wei

[willemdh]

Thanks @wei.wang for your answer.

Good to know ML - Kibana Space integration is on the roadmap.

My point of issue 2 was that I don't see any annotation at all, although there is definitely an anomaly. (Sry, I should have opened this in APM section)

The issue https://github.com/elastic/kibana/issues/52250 seems more about adding the annotations and plot tot the legend?

[wei.wang]

Hmm, yes, an annotation should be seen on your "APM - Transaction duration" since the anomaly score is 88 which is above 75. Please ask this in APM channel, where you can get better answer.