Machine Learning for Visualizations

[pianist_iris]

We have multiple Kibana visualizations which are traffic volume graph or login count graph.
Now we would like to utilize machine learning module to detect anomaly in those graphs.
What would be the most efficient way to achieve this?

I probably could create multiple jobs manually, applying the same logic in each visualize into each job. Just wondering if there's any smarter way - both for the creation now, and for future maintenance across visualizations and machine learning jobs.

It would be so cool if there's a setting button next to each visualization, where we could just enable Machine Learning for that visualization!

Thanks!

[richcollier]

Hi - yes it would be really cool to ML-ify a data visualization (certainly if that visualization made sense and could be translated to an anomaly detection job config).

In the meantime, you'll have to "translate" the config of the visualization to the ML job config. For example, if the visualization shows the count of something over time (i.e. a login count graph) then you will use the count function of ML. If you have a metric-based graph (i.e. sum of network bytes) then you'd use sum(bytes).

If you have a "split" in your visualization ("Break down by") then this field could be a partition_field for the ML job.

So, as an example, here's a Lens visualization of some web logs looking at the sum(bytes) for each agent.keyword

The equivalent ML job for this (using the multi-metric wizard) would look like:

Essentially, exactly the same logic!

[richcollier]

Results!

[ghudgins (Graham Hudgins)]

Very interesting idea. Would be good to articulate it as a "Feature Request" in Kibana if you're up for it. Issues · elastic/kibana · GitHub.

As a complete aside we're also thinking about surfacing visualizations of anomalies and ML annotations in more places in Kibana...but that's the reverse of what you're suggesting. With Lens' multiple layers we do want to make it really easy to add ML Anomalies & Annotations as an overlay to your visualization.

Anyway - would love to see an issue for this to continue the discussion!!