Change ML instance configuration

Elastic Stack Elasticsearch
1

Not sure if this can be routed to a more appropriate space but how do we change the ML instance for Elastic Cloud deployments? For my eland experiment, I am currently assigned "aws.es.ml.c5d" (costly) but I'd like to change it to "aws.es.ml.m6gd". In my search to change instance configuration, I'm led to this doc that requires me to login to a URL I can't find.

Thanks!

2

Hi @Elijah_Adeoye welcome to the community! And thanks for using Elastic Cloud

First to be precise I think you using Elastic Cloud or Hosted Solution

Which can be accessed

https://cloud.elastic.co

The document you linked is to a different self hosted product and yes the names are a bit confusing... :frowning:

If you are using the hosted solution the please reference these documents

In elastic cloud the ML node choices are optimized HW profiles and should the best cost per performance for ML Use cases.

Today it looks like the c5d is the optimized profile today.

Now just to check you can change the profiles of the hot data nodes

Storage i3
Storage Dense i3en
General Purpose m6gd
CPU Optimize c6gd

The pricing of elastic components is not directly linear between data nodes and ML nodes, ML is a high value component perhaps that is the difference you are seeing

Typically depending on your use case the ML nodes are relatively smaller than the Data Nodes capacity. etc

Hope this helps a bit, let us know...

3

Thanks @stephenb . You're right - I am referring to elastic cloud. So to confirm, I cannot change the instance type for my ML nodes on elastic cloud?

4

Apologies but No ... Not today....

What size node are you thinking you need? If you can share your entire cluster config we can take a look.

Do you know what kind of ML jobs are going to run... When you set them up it'll estimate the amount of memory you need

5

Hello @stephenb . 4gb. See below please. It's an NLP job for semantic search. The model 's required_native_memory_bytes is 739.5MB though it's only about 249MB. Would you like me to share the "Equivalent API request" (hope that's safe but it's totally a testing environment anyway)? Frankly, the cheapest aws.es.ml.xxx, e.g. aws.es.ml.m6gd, will do.
image

6

Hi @Elijah_Adeoye

Perhaps use a 2gb instance or even 1gb

There's no way to just simply change to a different underlying hardware. This is the profile that elastic cloud uses for machine learning for thousands of customers at this time, Perhaps that will change in the future.

Also unless You are using searchable snapshots and frozen which you are probably not make sure that you are at platinum level subscription not enterprise.

There's not a direct mapping between AWS cost and elastic cloud cost... There is relationship but it's not necessarily one to one as there is value and software etc.

Hopefully you find enough value to continue to use Elastic Cloud

7

@stephenb . Thank you! I'm using a 4GB instance but no problem. I'll keep tinkering and we'll use elastic cloud for now. Thanks!

1 Like