# The E5 multilingual optimized returns a python error. (self hosted?)

**URL:** <https://discuss.elastic.co/t/the-e5-multilingual-optimized-returns-a-python-error-self-hosted/356157>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-machine-learning, docker\
**Created:** [March 26, 2024, 9:26am UTC](https://discuss.elastic.co/t/the-e5-multilingual-optimized-returns-a-python-error-self-hosted/356157 "2024-03-26T09:26:46Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Chenko](https://avatars.discourse-cdn.com/v4/letter/c/258eb7/32.png) [@Chenko](https://discuss.elastic.co/u/Chenko)\
**Post date:** [March 26, 2024, 9:26am UTC](https://discuss.elastic.co/t/the-e5-multilingual-optimized-returns-a-python-error-self-hosted/356157/1 "2024-03-26T09:26:46Z")

</div>

Hi,

I am trying to run the E5 multilingual optimized model myself, as I feel this might be faster than running it through elastic that is in a docker container.

I am running into the following issue:

When loading in the model with the following python line:  
` model = SentenceTransformer('elastic/multilingual-e5-small-optimized')`

I get the following error:

```auto
Loading model...
No sentence-transformers model found with name elastic/multilingual-e5-small-optimized. Creating a new one with MEAN pooling.
config.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 672/672 [00:00<00:00, 2.24MB/s]
pytorch_model.bin: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 412M/412M [00:36<00:00, 11.2MB/s]
Traceback (most recent call last):
  File "/Users/chenko/Documents/ELSER/e5_sentence_run/.venv/lib/python3.11/site-packages/transformers/modeling_utils.py", line 532, in load_state_dict
    return torch.load(
           ^^^^^^^^^^^
  File "/Users/chenko/Documents/ELSER/e5_sentence_run/.venv/lib/python3.11/site-packages/torch/serialization.py", line 1025, in load
    raise pickle.UnpicklingError(UNSAFE_MESSAGE + str(e)) from None
_pickle.UnpicklingError: Weights only load failed. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution.Do it only if you get the file from a trusted source. WeightsUnpickler error: Unsupported class torch.qint8

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/chenko/Documents/ELSER/e5_sentence_run/.venv/lib/python3.11/site-packages/transformers/modeling_utils.py", line 541, in load_state_dict
    if f.read(7) == "version":
       ^^^^^^^^^
  File "<frozen codecs>", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 128: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/chenko/Documents/ELSER/e5_sentence_run/main.py", line 54, in <module>
    main()
  File "/Users/chenko/Documents/ELSER/e5_sentence_run/main.py", line 24, in main
    model = SentenceTransformer('elastic/multilingual-e5-small-optimized')
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chenko/Documents/ELSER/e5_sentence_run/.venv/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 199, in __init__
    modules = self._load_auto_model(
              ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chenko/Documents/ELSER/e5_sentence_run/.venv/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 1134, in _load_auto_model
    transformer_model = Transformer(
                        ^^^^^^^^^^^^
  File "/Users/chenko/Documents/ELSER/e5_sentence_run/.venv/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 36, in __init__
    self._load_model(model_name_or_path, config, cache_dir, **model_args)
  File "/Users/chenko/Documents/ELSER/e5_sentence_run/.venv/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 65, in _load_model
    self.auto_model = AutoModel.from_pretrained(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chenko/Documents/ELSER/e5_sentence_run/.venv/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 563, in from_pretrained
    return model_class.from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chenko/Documents/ELSER/e5_sentence_run/.venv/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3335, in from_pretrained
    state_dict = load_state_dict(resolved_archive_file)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/chenko/Documents/ELSER/e5_sentence_run/.venv/lib/python3.11/site-packages/transformers/modeling_utils.py", line 553, in load_state_dict
    raise OSError(
OSError: Unable to load weights from pytorch checkpoint file for '/Users/chenko/.cache/huggingface/hub/models--elastic--multilingual-e5-small-optimized/snapshots/2612fa238fbd4d6348b6aef85906b3d3d3a8fec3/pytorch_model.bin' at '/Users/chenko/.cache/huggingface/hub/models--elastic--multilingual-e5-small-optimized/snapshots/2612fa238fbd4d6348b6aef85906b3d3d3a8fec3/pytorch_model.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.

```

As can be seen it does download the model, however it says the following:

```auto
No sentence-transformers model found with name elastic/multilingual-e5-small-optimized. Creating a new one with MEAN pooling.

```

However when trying this with the original e5 model: `intfloat/multilingual-e5-small` it works perfectly fine.

Any help would be appreciated!

Kind regards, Chenko

---

<div class="post-metadata">

**Author:** ![valeriy42](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/valeriy42/32/82758_2.png) [@valeriy42](https://discuss.elastic.co/u/valeriy42)\
**Post date:** [April 8, 2024, 7:34am UTC](https://discuss.elastic.co/t/the-e5-multilingual-optimized-returns-a-python-error-self-hosted/356157/2 "2024-04-08T07:34:46Z")

</div>

Hello @Chenko,

Our optimized version of `multilingual-e5-small` is a quantized and traced version. It won't be loaded with `SentenceTransformer`. It is supposed to be used with Elasticsearch in [an air-gapped environment](https://www.elastic.co/guide/en/machine-learning/current/ml-nlp-e5.html#_using_the_huggingface_repository).

Regards,  
Valeriy

---

<div class="post-metadata">

**Author:** ![Chenko](https://avatars.discourse-cdn.com/v4/letter/c/258eb7/32.png) [@Chenko](https://discuss.elastic.co/u/Chenko)\
**Post date:** [April 8, 2024, 8:08am UTC](https://discuss.elastic.co/t/the-e5-multilingual-optimized-returns-a-python-error-self-hosted/356157/3 "2024-04-08T08:08:34Z")

</div>

Thanks @valeriy42 ,

How would it possible to run this model outside of elastic?  
If at all?

Kind regards,  
Chenko

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 6, 2024, 8:09am UTC](https://discuss.elastic.co/t/the-e5-multilingual-optimized-returns-a-python-error-self-hosted/356157/4 "2024-05-06T08:09:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

---

<div class="post-metadata">

**Author:** ![valeriy42](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/valeriy42/32/82758_2.png) [@valeriy42](https://discuss.elastic.co/u/valeriy42)\
**Post date:** [May 13, 2024, 11:03am UTC](https://discuss.elastic.co/t/the-e5-multilingual-optimized-returns-a-python-error-self-hosted/356157/5 "2024-05-13T11:03:31Z")

</div>

Hello @Chenko ,

we don't support running this model outside of the Elasticsearch.

Regards,  
Valeriy