Which Machine learning Algorithms used by Elastic x-pack ML?

Elastic Stack Elasticsearch
1

Hello,
i have 2 questions:

1- which Machine learning Algorithms used by x-pack Machine Learning?

2- What do you suggest if I want to use other Machine Learning Algorithms for elatstic data? should i use MLlib spark on Elasticsearch data ??

Thanks for your help
Ali

2

The technology behind Elastic’s X-Pack Machine Learning (ML) is a bespoke amalgamation of different machine learning methods and techniques that brings sophisticated real-time automated anomaly detection for time series data to users that may not be able to employ data science on their own. Using techniques such as clustering, various types of time-series decomposition, Bayesian distribution modeling, and correlation analysis, X-Pack Machine Learning takes a 100% unsupervised machine learning approach to statistically model data’s time-based characteristics merely by observing its historical behavior. Behind the scenes, a dynamic, ever-learning statistical model is built and stored, per unique time-series. Real-time data being analyzed both contributes to this model’s maturity and is assessed against the model so that it can be judged for its level of unusualness. If the data’s behavior is seen as being within the low probability range, an anomaly record is created, persisted, and scored proportional to the probability. This score is normalized on a user-friendly dynamic scale between 0 and 100, where 100 is the most unusual thing ever detected for the data set. This makes it easy to interface ML to Alerting, allowing the user to rate-control alerting based upon a desired severity of anomaly’s unusualness. A full featured UI, implemented as a Kibana plug-in, allows the user the capability to define, control, execute, and report on ML’s analysis of different data sets. A full-featured API also allows ML to be programmatically controled.

In summary:

  • Automated analysis of data using unsupervised machine learning
  • Data’s behavior is automatically learned simply by observation (“baselined”)
  • Statistical models are created for every time series/entity
  • Anomalous behavior is automatically found and scored empirically
4 Likes