What benefits do I get from Elasticsearch's paid ML features?

[green12]

Hi everyone.

I'm planning to implement several machine learning features in my Elasticsearch-based app, and it turns out that ML features are only supported starting from the Platinum plan (I'm currently using the free plan as I'm not using Elastic cloud).

After reading some documentation, it seems like the ML features supported in the plan only provide the ability to import a pre-trained model into the cluster, and then use the cluster as some kind of an API to make an inference using the model (For example in the case of Named Entity Recognition and Text Embedding).

My question is, wouldn't I be able to achieve the same thing if I just deploy my model outside of Elasticsearch (Since it's basically an API)? Or is there any other benefits I can get by purchasing the Platinum plan that could help me in my machine learning implementation?

Thanks in advance!

[grabowskit (Tom Grabowski)]

HI @green12

With the Platinum subscription tier you get access to all the ML features, including Unsupervised ML - Anomaly Detection & Outlier Detection, Training Supervised ML models - Data frame analysis for Classification and Regression, the ability to import 3rd party models and use Inference ingest processor, plus access to all the ML APIs and Kibana ML app UI and access to Elastic support.

It would be helpful to hear more about your app and what your specific ML features you are planning on implementing. You mentioned in your question using NER and Text embedding, which would require the licensed feature to import PyTorch NLP models and use them in an Inference ingest processor. Model Management can be done through the Kibana ML UI or via Elasticsearch APIs.

To answer the question in the last paragraph about the benefit of using ML models within Elasticsearch rather than an external API, the main reasons is usually around reducing the cost of moving the data in and out of Elasticsearch (through external models), improving model scalability and security within Elasticsearch, and reducing the complexity and supportability by not having different products for data store and ml model management.

Hope that helps. Thanks for the question!

Tom

[green12]

I see. I'll take these into considerations.

Thank you so much for the explanation!