ML node memory configuration

[danielyahn]

From Sizing for Machine Learning with Elasticsearch | Elastic Blog

We recommend that ML nodes have at least 4 Cores and 64GB RAM. Assign 24GB of memory to the JVM Heap, reserving 40GB for ML processing.

Is it possible to configure ML-dedicated node from ECE with such heap and off-heap requirements?

[richcollier]

I don't believe so. According to https://www.elastic.co/guide/en/cloud-enterprise/2.4/ece-heap.html, ECE automatically allocates %50 of the available RAM to the JVM and there doesn't seem to be a way for the user to modify that.

[danielyahn]

Thanks for the input. Any other guidance on how to size the ML node? Other than:

64GB RAM. Assign 24GB of memory to the JVM Heap, reserving 40GB for ML processing.

[richcollier]

ML jobs are almost always "constrained" by the amount of memory required to run - and that's a function of how many unique time-series being modeled. In general, the conservative rule of thumb is a 100MB base overhead, plus 20k-30k of RAM for every partition field. So, a job running on 5000 unique time-series could require about 250MB of RAM.

The default "limit" of RAM per ML job is 1024mb as set by the parameter model_memory_limit (see docs)