# Hi, I am trying to deploy the Elser (built-in) model, and all the configurations seems to be fine. But the deployment has started and it takes forever to complete the deployment
**URL:** https://discuss.elastic.co/t/hi-i-am-trying-to-deploy-the-elser-built-in-model-and-all-the-configurations-seems-to-be-fine-but-the-deployment-has-started-and-it-takes-forever-to-complete-the-deployment/342350
**Category:** Elasticsearch
**Tags:** elastic-stack-machine-learning
**Created:** [September 5, 2023, 2:11pm UTC](https://discuss.elastic.co/t/hi-i-am-trying-to-deploy-the-elser-built-in-model-and-all-the-configurations-seems-to-be-fine-but-the-deployment-has-started-and-it-takes-forever-to-complete-the-deployment/342350 "2023-09-05T14:11:53Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@Manasa4](https://discuss.elastic.co/u/Manasa4)
#### Post date: [September 5, 2023, 2:11pm UTC](https://discuss.elastic.co/t/hi-i-am-trying-to-deploy-the-elser-built-in-model-and-all-the-configurations-seems-to-be-fine-but-the-deployment-has-started-and-it-takes-forever-to-complete-the-deployment/342350/1 "2023-09-05T14:11:53Z")
In my case I have run the deployment at 11:27AM EST on 30th of August , but till 11:30 AM EST 31st of August, it still seems to be running.
Also, tried increasing the RAM and space for Elastic search and also for the ML server, but nothing worked.
Also, the same thing happened for the key phrase extraction hugging face model.
Can someone please suggest me on this?
---
### Author:  [@wei.wang](https://discuss.elastic.co/u/wei.wang)
#### Post date: [September 6, 2023, 2:23am UTC](https://discuss.elastic.co/t/hi-i-am-trying-to-deploy-the-elser-built-in-model-and-all-the-configurations-seems-to-be-fine-but-the-deployment-has-started-and-it-takes-forever-to-complete-the-deployment/342350/2 "2023-09-06T02:23:22Z")
> [@Manasa4](#):
>
> nothing worked.
Hi,
Can you please write down the steps you have tried, and describe which part is not working. A few screenshots would be helpful too.
Thanks.
---
### Author:  [@Manasa4](https://discuss.elastic.co/u/Manasa4)
#### Post date: [September 6, 2023, 9:17pm UTC](https://discuss.elastic.co/t/hi-i-am-trying-to-deploy-the-elser-built-in-model-and-all-the-configurations-seems-to-be-fine-but-the-deployment-has-started-and-it-takes-forever-to-complete-the-deployment/342350/3 "2023-09-06T21:17:18Z")
Hi Wang,
Thanks for reaching out. I'm unable to post any of the screenshots and to mention my issue, I have downloaded the elser built in model
from the Machine Learning -\> Trained Model section and once after it's downloaded, I tried deploying by clicking on "Start deployment" option by setting the **"Number of Allocations to 1" and "Threads per Allocation to 1"**
It was running for 1 complete day i.e ; 24 hours(From 11:27AM EST on 30th of August until 11:30 AM EST 31st of August) and I manually tried to stop the deployment as it was taking so much time.
The configurations used while deploying the elser model was :
**Elastic search space and RAM : 90 GB Storage | 2GB RAM | Upto 2.5 vCPU and Availability zones are set to 3**
**Machine Learning node space and RAM : 4 GB RAM|2 vCPU Upto 8 vCPU and Availability zones are set to 3**
Please let me know if you need any further information and also if there is any resolution, please let me know the same.
Thanks,
Manasa.
---
### Author:  [@wei.wang](https://discuss.elastic.co/u/wei.wang)
#### Post date: [September 7, 2023, 12:52am UTC](https://discuss.elastic.co/t/hi-i-am-trying-to-deploy-the-elser-built-in-model-and-all-the-configurations-seems-to-be-fine-but-the-deployment-has-started-and-it-takes-forever-to-complete-the-deployment/342350/4 "2023-09-07T00:52:36Z")
Hi Manasa,
> [@Manasa4](#):
>
> It was running for 1 complete day
did you see the below message after you started deployment? and what is the state shown on your screen?
You also can check ml node state from Machine Learning -\> Memory Usage page, or sending this request `GET _ml/trained_models/.elser_model_1/_stats` through dev tools console.
thanks.
---
### Author:  [@Manasa4](https://discuss.elastic.co/u/Manasa4)
#### Post date: [September 7, 2023, 3:35am UTC](https://discuss.elastic.co/t/hi-i-am-trying-to-deploy-the-elser-built-in-model-and-all-the-configurations-seems-to-be-fine-but-the-deployment-has-started-and-it-takes-forever-to-complete-the-deployment/342350/5 "2023-09-07T03:35:09Z")
Yes! I did see the message saying "Deployment has started" for the respective elser\_model.
The stats were also showing it's running with 0 errors.
Not sure, why is it taking that long time because tere are no errors comping up while the deployment is going on.
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [September 7, 2023, 5:28am UTC](https://discuss.elastic.co/t/hi-i-am-trying-to-deploy-the-elser-built-in-model-and-all-the-configurations-seems-to-be-fine-but-the-deployment-has-started-and-it-takes-forever-to-complete-the-deployment/342350/6 "2023-09-07T05:28:34Z")
> [@Manasa4](#):
>
> Not sure, why is it taking that long time because tere are no errors comping up while the deployment is going on.
Silly question ... what exactly is taking a long time are you running an inference pipeline?
Once the ELSER model is deployed and started (as shown above) it does not do anything until you use the inference pipeline to do the text expansion.
Have you created a pipeline and then posted documents through it?
If you are interested here is an end-to-end Notebook that you can try.
P.S. I am `bvader`
>
---
### Author:  [@Manasa4](https://discuss.elastic.co/u/Manasa4)
#### Post date: [September 7, 2023, 12:55pm UTC](https://discuss.elastic.co/t/hi-i-am-trying-to-deploy-the-elser-built-in-model-and-all-the-configurations-seems-to-be-fine-but-the-deployment-has-started-and-it-takes-forever-to-complete-the-deployment/342350/7 "2023-09-07T12:55:21Z")
The deployment itself is not happening. It's taking forever to complete.
---
### Author:  [@Manasa4](https://discuss.elastic.co/u/Manasa4)
#### Post date: [September 7, 2023, 1:25pm UTC](https://discuss.elastic.co/t/hi-i-am-trying-to-deploy-the-elser-built-in-model-and-all-the-configurations-seems-to-be-fine-but-the-deployment-has-started-and-it-takes-forever-to-complete-the-deployment/342350/8 "2023-09-07T13:25:34Z")
And the above issue still continues.
But just to check if it's the same thing happening on the other cluster too, I have created a new cluster and tried deploying the elser model with these configurations :
**"Number of Allocations to 1" and "Threads per Allocation to 1"**
**Elastic search space and RAM : 90 GB Storage | 2GB RAM | Upto 2.5 vCPU and Availability zones are set to 3**
**Machine Learning node space and RAM : 4 GB RAM|2 vCPU Upto 8 vCPU and Availability zones are set to 3**
And I'm facing the below error :
The elser model is again failing with this issue on the new cluster :
{ "statusCode": 429, "error": "Too Many Requests", "message": "[circuit\_breaking\_exception\n\tRoot causes:\n\t\tcircuit\_breaking\_exception: [parent] Data too large, data for [\] would be [1052402776/1003.6mb], which is larger than the limit of [1020054732/972.7mb], real usage: [1052402776/1003.6mb], new bytes reserved: [0/0b], usages [model\_inference=518432/506.2kb, eql\_sequence=0/0b, fielddata=0/0b, request=0/0b, inflight\_requests=0/0b]]: [parent] Data too large, data for [\] would be [1052402776/1003.6mb], which is larger than the limit of [1020054732/972.7mb], real usage: [1052402776/1003.6mb], new bytes reserved: [0/0b], usages [model\_inference=518432/506.2kb, eql\_sequence=0/0b, fielddata=0/0b, request=0/0b, inflight\_requests=0/0b]", "attributes": { "body": { "error": { "root\_cause": [{ "type": "circuit\_breaking\_exception", "reason": "[parent] Data too large, data for [\] would be [1052402776/1003.6mb], which is larger than the limit of [1020054732/972.7mb], real usage: [1052402776/1003.6mb], new bytes reserved: [0/0b], usages [model\_inference=518432/506.2kb, eql\_sequence=0/0b, fielddata=0/0b, request=0/0b, inflight\_requests=0/0b]", "bytes\_wanted": 1052402776, "bytes\_limit": 1020054732, "durability": "TRANSIENT" } ], "type": "circuit\_breaking\_exception", "reason": "[parent] Data too large, data for [\] would be [1052402776/1003.6mb], which is larger than the limit of [1020054732/972.7mb], real usage: [1052402776/1003.6mb], new bytes reserved: [0/0b], usages [model\_inference=518432/506.2kb, eql\_sequence=0/0b, fielddata=0/0b, request=0/0b, inflight\_requests=0/0b]", "bytes\_wanted": 1052402776, "bytes\_limit": 1020054732, "durability": "TRANSIENT" }, "status": 429 } } }
Please help me on this.
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [September 7, 2023, 2:13pm UTC](https://discuss.elastic.co/t/hi-i-am-trying-to-deploy-the-elser-built-in-model-and-all-the-configurations-seems-to-be-fine-but-the-deployment-has-started-and-it-takes-forever-to-complete-the-deployment/342350/9 "2023-09-07T14:13:03Z")
> [@Manasa4](#):
>
> Yes! I did see the message saying "Deployment has started" for the respective elser\_model.
Hi @Manasa4
Can you provide a screenshot please of that trained model screen.
What command did you run resulted in the error above?
Is it when you tried to load a document through in the inference pipeline?
Did you try to load an extremely large document?
That error basically says the request exceeded that JVM heap.
Is there any other activity going on on that cluster? Is it busy with many other tasks?
I would try making the Data Nodes 4GB
You could also try setting the Allocations and threads both to 2 when you load the model.
---
### Author:  [@dkyle](https://discuss.elastic.co/u/dkyle)
#### Post date: [September 11, 2023, 9:16am UTC](https://discuss.elastic.co/t/hi-i-am-trying-to-deploy-the-elser-built-in-model-and-all-the-configurations-seems-to-be-fine-but-the-deployment-has-started-and-it-takes-forever-to-complete-the-deployment/342350/10 "2023-09-11T09:16:43Z")
@Manasa4 thanks for reporting the problem, I've opened an issue in the Elasticsearch repository to track the issue.
>
>
> \### Elasticsearch Version
>
> 8.9.1
>
> \### Installed Plugins
>
> \_No response\_
>
> \### Java… Version
>
> \_bundled\_
>
> \### OS Version
>
> any
>
> \### Problem Description
>
> Deploying the ELSER model can cause a \`CircuitBreakingException\` if the cluster is being used heavily. Often the \`CircuitBreakingException\` is thrown because a request from Kibana or another client cannot be serviced with the available memory while loading the ELSER model which temporarily increases memory pressure in the JVM. In this scenario Kibana becomes unresponsive but in many cases the ELSER model deployment completes successfully but the UI takes a while to see that status change.
>
> \`\`\`
> Caused by: org.elasticsearch.common.breaker.CircuitBreakingException: \[parent\] Data too large, data for \[indices:data/read/search\[phase/query\]\] would be \[1045092822/996.6mb\], which is larger than the limit of \[1020054732/972.7mb\], real usage: \[1045091992/996.6mb\], new bytes reserved: \[830/830b\], usages \[eql\_sequence=0/0b, fielddata=1779/1.7kb, request=0/0b, inflight\_requests=830/830b, model\_inference=0/0b\] at org.elasticsearch.indices.breaker.HierarchyCircuitBreakerService.checkParentLimit(HierarchyCircuitBreakerService.java:414)
> \`\`\`
>
> \### Steps to Reproduce
>
> The problem occurs more commonly on smaller nodes with less JVM heap memory (e.g. 2GB JVM heap). Reproduce by repeatedly deploying and un-deploying the ELSER model.
>
> \### Logs (if relevant)
>
> \_No response\_
We are working on a fix, in the meantime I would do as @stephenb suggested and increase the data node size to 4GB.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 9, 2023, 9:17am UTC](https://discuss.elastic.co/t/hi-i-am-trying-to-deploy-the-elser-built-in-model-and-all-the-configurations-seems-to-be-fine-but-the-deployment-has-started-and-it-takes-forever-to-complete-the-deployment/342350/11 "2023-10-09T09:17:09Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.