Unable to load a trained model on ML node with sufficient memory

Elastic Stack Elasticsearch
1

Hello

Here is the scenario.

I have 2 ML nodes with 8 procs/64 Gb on Elastic 8.15.3.

I start by loading a large bge_m3 model with 1 allocation and 1 thread. :slight_smile:

image
image1628×809 50.6 KB

As you can see on picture, one node had a lot of memory consumed but the second node had no model deployed to it with all the memory free.

I try to load a smaller model like bge-large-en-v1.5 or multilingual-e5-large. Obviously, it should be able to run on the second node.

But insted, I have this error :

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "not enough memory on node [sCw90xbzTs6faJw26Rozog] to assign model [baai_bge_m3]"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "not enough memory on node [sCw90xbzTs6faJw26Rozog] to assign model [baai_bge_m3]"
  },
  "status": 400
}

What is happening here? Is Elastic not seeing my second node? I don't see what configuration to change to alter this behavior.

Regards