# Automate ELSER v2 Trained Models download

**URL:** <https://discuss.elastic.co/t/automate-elser-v2-trained-models-download/355177>\
**Category:** Elasticsearch\
**Created:** [March 11, 2024, 6:03pm UTC](https://discuss.elastic.co/t/automate-elser-v2-trained-models-download/355177 "2024-03-11T18:03:34Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![GlebCA](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/glebca/32/78044_2.png) [@GlebCA](https://discuss.elastic.co/u/GlebCA)\
**Post date:** [March 11, 2024, 6:03pm UTC](https://discuss.elastic.co/t/automate-elser-v2-trained-models-download/355177/1 "2024-03-11T18:03:34Z")

</div>

Hi,

I need to automate ELSER v2 Machine Learning Trained Models download into Elastic for integration testing purposes.

There is documentation on how to download ELSER v2 from local files [https://www.elastic.co/guide/en/machine-learning/current/ml-nlp-elser.html#\_using\_file\_based\_access](https://www.elastic.co/guide/en/machine-learning/current/ml-nlp-elser.html#_using_file_based_access). But it requires some manual operations in Kibana (Machine Learning -\> Trained Models -\> Add trained model -\> Download)

I'd like to download ELSER v2 files to local file system and automatically download this Model into Elastic without any interaction with Kibana for integration testing purposes.

I reviewed available APIs [https://www.elastic.co/guide/en/elasticsearch/reference/current/put-trained-models.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/put-trained-models.html) and was able to compose "Create trained models API", but the model cannot be started because index with model dictionary is not initialized during this API call.

Could you please suggest how it is possible to automatically download and deploy ELSER v2 into Elastic?

Thank you in advance,  
Gleb

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [March 11, 2024, 9:07pm UTC](https://discuss.elastic.co/t/automate-elser-v2-trained-models-download/355177/2 "2024-03-11T21:07:10Z")

</div>

Hi @GlebCA Welcome to the community

I _ **think** _ everything you are looking for is [here](https://www.elastic.co/guide/en/machine-learning/8.12/ml-nlp-elser.html#dev-console)

I have not loaded from a filesystem but I just ran these and it worked

Downloaded and start.

There is also python libraries for this beside just pure `curl`

```auto
sbrown$ curl -X PUT "localhost:9200/_ml/trained_models/.elser_model_2?pretty" -H 'Content-Type: application/json' -d'{ "input": { "field_names": ["text_field"] }}'

{
  "model_id" : ".elser_model_2",
  "model_type" : "pytorch",
  "model_package" : {
    "packaged_model_id" : "elser_model_2",
    "model_repository" : "https://ml-models.elastic.co",
    "minimum_version" : "11.0.0",
    "size" : 438123914,
    "sha256" : "2e0450a1c598221a919917cbb05d8672aed6c613c028008fedcd696462c81af0",
    "metadata" : { },
    "tags" : [],
    "vocabulary_file" : "elser_model_2.vocab.json"
  },
  "created_by" : "api_user",
  "version" : "12.0.0",
  "create_time" : 1710191067705,
  "model_size_bytes" : 0,
  "estimated_operations" : 0,
  "license_level" : "platinum",
  "description" : "Elastic Learned Sparse EncodeR v2",
  "tags" : [
    "elastic"
  ],
  "metadata" : { },
  "input" : {
    "field_names" : [
      "text_field"
    ]
  },
  "inference_config" : {
    "text_expansion" : {
      "vocabulary" : {
        "index" : ".ml-inference-native-000002"
      },
      "tokenization" : {
        "bert" : {
          "do_lower_case" : true,
          "with_special_tokens" : true,
          "max_sequence_length" : 512,
          "truncate" : "first",
          "span" : -1
        }
      }
    }
  },
  "location" : {
    "index" : {
      "name" : ".ml-inference-native-000002"
    }
  }
}

hyperion:docker sbrown$ 

hyperion:docker sbrown$ curl -X POST "localhost:9200/_ml/trained_models/.elser_model_2/deployment/_start?deployment_id=for_search&pretty"
{
  "assignment" : {
    "task_parameters" : {
      "model_id" : ".elser_model_2",
      "deployment_id" : "for_search",
      "model_bytes" : 438123914,
      "threads_per_allocation" : 1,
      "number_of_allocations" : 1,
      "queue_capacity" : 1024,
      "cache_size" : "438123914b",
      "priority" : "normal",
      "per_deployment_memory_bytes" : 0,
      "per_allocation_memory_bytes" : 0
    },
    "routing_table" : {
      "xrWfSmBySNaLakwl5N46XQ" : {
        "current_allocations" : 1,
        "target_allocations" : 1,
        "routing_state" : "started",
        "reason" : ""
      }
    },
    "assignment_state" : "started",
    "start_time" : "2024-03-11T21:05:36.439254475Z",
    "max_assigned_allocations" : 1
  }
}
hyperion:docker sbrown$

```

---

<div class="post-metadata">

**Author:** ![GlebCA](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/glebca/32/78044_2.png) [@GlebCA](https://discuss.elastic.co/u/GlebCA)\
**Post date:** [March 12, 2024, 2:38am UTC](https://discuss.elastic.co/t/automate-elser-v2-trained-models-download/355177/3 "2024-03-12T02:38:15Z")

</div>

Great, thank you for the quick help!

The model was deployed successfully. The only concern I have is a message in Kibana -\> Machine Learning -\> Trained Models:

> ML job and trained model synchronization required  
> Some jobs or trained models are missing or have incomplete saved objects. Synchronize your jobs and trained models.

It is necessary to synchronize? Can synchronization be done via API?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [March 12, 2024, 3:27am UTC](https://discuss.elastic.co/t/automate-elser-v2-trained-models-download/355177/4 "2024-03-12T03:27:19Z")

</div>

I think that is a UI thing  
I can tell you I have loaded and used the models without ever logging into Kibana.

I have a slightly dated Collab notebook that does everything... No Kibana.

> <https://github.com/bvader/elasticsearch-test-elser/blob/main/elasticsearch_test_elser.ipynb>

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 9, 2024, 3:27am UTC](https://discuss.elastic.co/t/automate-elser-v2-trained-models-download/355177/5 "2024-04-09T03:27:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.