# Elastic search trained model inference not working **URL:** \ **Category:** Elasticsearch\ **Tags:** docker\ **Created:** [May 16, 2024, 3:40pm UTC](https://discuss.elastic.co/t/elastic-search-trained-model-inference-not-working/359649 "2024-05-16T15:40:31Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![likealam](https://avatars.discourse-cdn.com/v4/letter/l/9fc348/32.png) [@likealam](https://discuss.elastic.co/u/likealam)\ **Post date:** [May 16, 2024, 3:40pm UTC](https://discuss.elastic.co/t/elastic-search-trained-model-inference-not-working/359649/1 "2024-05-16T15:40:31Z")
After importing the text embedding sentence-transformers/msmarco-MiniLM-L12-cos-v5 model to elasticsearch using eland the model infering doesnt work. eland command: `eland_import_hub_model --url http://localhost:9200 --hub-model-id sentence-transformers/msmarco-MiniLM-L-12-v3 --task-type text_embedding --start` Infer request: ```json POST /_ml/trained_models/sentence-transformers__msmarco-minilm-l12-cos-v5/deployment/_infer { "docs": { "text_field": "How is the weather in Jamaica?" } } ``` Error: ```json { "error": { "root_cause": [ { "type": "status_exception", "reason": "Error in inference process: [inference canceled as process is stopping]" } ], "type": "status_exception", "reason": "Error in inference process: [inference canceled as process is stopping]" }, "status": 500 } ``` Docker logs of all the processes ```auto 2024-05-16 14:47:29 {"@timestamp":"2024-05-16T12:47:29.207Z", "log.level": "INFO", "message":"[sentence-transformers__msmarco-minilm-l12-cos-v5] Starting model deployment of model [sentence-transformers__msmarco-minilm-l12-cos-v5]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[f059c181c171][ml_utility][T#3]","log.logger":"org.elasticsearch.xpack.ml.inference.deployment.DeploymentManager","elasticsearch.cluster.uuid":"eEqzvIFnQx6Lt08J4545wQ","elasticsearch.node.id":"-It6EisaTT2HVi05RqZGdA","elasticsearch.node.name":"f059c181c171","elasticsearch.cluster.name":"elasticsearch"} 2024-05-16 14:47:55 {"@timestamp":"2024-05-16T12:47:55.102Z", "log.level": "WARN", "data_stream.dataset":"deprecation.elasticsearch","data_stream.namespace":"default","data_stream.type":"logs","elasticsearch.event.category":"api","event.code":"deprecated_route_POST_/_ml/trained_models/{model_id}/deployment/_infer","message":"[POST /_ml/trained_models/{model_id}/deployment/_infer] is deprecated! Use [POST /_ml/trained_models/{model_id}/_infer] instead." , "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"deprecation.elasticsearch","process.thread.name":"elasticsearch[f059c181c171][transport_worker][T#4]","log.logger":"org.elasticsearch.deprecation.rest.RestController","elasticsearch.cluster.uuid":"eEqzvIFnQx6Lt08J4545wQ","elasticsearch.node.id":"-It6EisaTT2HVi05RqZGdA","elasticsearch.node.name":"f059c181c171","elasticsearch.cluster.name":"elasticsearch"} 2024-05-16 14:47:55 {"@timestamp":"2024-05-16T12:47:55.413Z", "log.level":"ERROR", "message":"[sentence-transformers__msmarco-minilm-l12-cos-v5] pytorch_inference/336 process stopped unexpectedly: Fatal error: 'si_signo 11, si_code: 1, si_errno: 0, address: 0xffffb27ca140, library: /lib/aarch64-linux-gnu/libc.so.6, base: 0xffffb26bd000, normalized address: 0x10d140', version: 8.13.4 (build 8480947324d752)\n", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[f059c181c171][ml_native_inference_comms][T#4]","log.logger":"org.elasticsearch.xpack.ml.process.AbstractNativeProcess","elasticsearch.cluster.uuid":"eEqzvIFnQx6Lt08J4545wQ","elasticsearch.node.id":"-It6EisaTT2HVi05RqZGdA","elasticsearch.node.name":"f059c181c171","elasticsearch.cluster.name":"elasticsearch"} 2024-05-16 14:47:55 {"@timestamp":"2024-05-16T12:47:55.414Z", "log.level":"ERROR", "message":"[sentence-transformers__msmarco-minilm-l12-cos-v5] inference process crashed due to reason [[sentence-transformers__msmarco-minilm-l12-cos-v5] pytorch_inference/336 process stopped unexpectedly: Fatal error: 'si_signo 11, si_code: 1, si_errno: 0, address: 0xffffb27ca140, library: /lib/aarch64-linux-gnu/libc.so.6, base: 0xffffb26bd000, normalized address: 0x10d140', version: 8.13.4 (build 8480947324d752)\n]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[f059c181c171][ml_native_inference_comms][T#4]","log.logger":"org.elasticsearch.xpack.ml.inference.deployment.DeploymentManager","elasticsearch.cluster.uuid":"eEqzvIFnQx6Lt08J4545wQ","elasticsearch.node.id":"-It6EisaTT2HVi05RqZGdA","elasticsearch.node.name":"f059c181c171","elasticsearch.cluster.name":"elasticsearch"} 2024-05-16 14:47:55 {"@timestamp":"2024-05-16T12:47:55.414Z", "log.level": "INFO", "message":"Inference process [sentence-transformers__msmarco-minilm-l12-cos-v5] failed due to [[sentence-transformers__msmarco-minilm-l12-cos-v5] pytorch_inference/336 process stopped unexpectedly: Fatal error: 'si_signo 11, si_code: 1, si_errno: 0, address: 0xffffb27ca140, library: /lib/aarch64-linux-gnu/libc.so.6, base: 0xffffb26bd000, normalized address: 0x10d140', version: 8.13.4 (build 8480947324d752)\n]. This is the [1] failure in 24 hours, and the process will be restarted.", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[f059c181c171][ml_native_inference_comms][T#4]","log.logger":"org.elasticsearch.xpack.ml.inference.deployment.DeploymentManager","elasticsearch.cluster.uuid":"eEqzvIFnQx6Lt08J4545wQ","elasticsearch.node.id":"-It6EisaTT2HVi05RqZGdA","elasticsearch.node.name":"f059c181c171","elasticsearch.cluster.name":"elasticsearch"} 2024-05-16 14:47:55 {"@timestamp":"2024-05-16T12:47:55.414Z", "log.level": "INFO", "message":"[sentence-transformers__msmarco-minilm-l12-cos-v5] Starting model deployment of model [sentence-transformers__msmarco-minilm-l12-cos-v5]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[f059c181c171][ml_native_inference_comms][T#4]","log.logger":"org.elasticsearch.xpack.ml.inference.deployment.DeploymentManager","elasticsearch.cluster.uuid":"eEqzvIFnQx6Lt08J4545wQ","elasticsearch.node.id":"-It6EisaTT2HVi05RqZGdA","elasticsearch.node.name":"f059c181c171","elasticsearch.cluster.name":"elasticsearch"} 2024-05-16 14:47:55 {"@timestamp":"2024-05-16T12:47:55.414Z", "log.level": "WARN", "message":"path: /_ml/trained_models/sentence-transformers __msmarco-minilm-l12-cos-v5/deployment/_infer, params: {model_id=sentence-transformers__ msmarco-minilm-l12-cos-v5}, status: 500", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[f059c181c171][ml_native_inference_comms][T#5]","log.logger":"rest.suppressed","elasticsearch.cluster.uuid":"eEqzvIFnQx6Lt08J4545wQ","elasticsearch.node.id":"-It6EisaTT2HVi05RqZGdA","elasticsearch.node.name":"f059c181c171","elasticsearch.cluster.name":"elasticsearch","error.type":"org.elasticsearch.ElasticsearchStatusException","error.message":"Error in inference process: [inference canceled as process is stopping]","error.stack_trace":"org.elasticsearch.ElasticsearchStatusException: Error in inference process: [inference canceled as process is stopping]\n\tat org.elasticsearch.ml@8.13.4/org.elasticsearch.xpack.ml.inference.deployment.AbstractPyTorchAction.onFailure(AbstractPyTorchAction.java:114)\n\tat org.elasticsearch.ml@8.13.4/org.elasticsearch.xpack.ml.inference.deployment.InferencePyTorchAction.processResult(InferencePyTorchAction.java:182)\n\tat org.elasticsearch.ml@8.13.4/org.elasticsearch.xpack.ml.inference.deployment.InferencePyTorchAction.lambda$doRun$3(InferencePyTorchAction.java:151)\n\tat org.elasticsearch.server@8.13.4/org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:171)\n\tat org.elasticsearch.ml@8.13.4/org.elasticsearch.xpack.ml.inference.pytorch.process.PyTorchResultProcessor.lambda$notifyAndClearPendingResults$3(PyTorchResultProcessor.java:145)\n\tat java.base/java.util.concurrent.ConcurrentHashMap.forEach(ConcurrentHashMap.java:1603)\n\tat org.elasticsearch.ml@8.13.4/org.elasticsearch.xpack.ml.inference.pytorch.process.PyTorchResultProcessor.notifyAndClearPendingResults(PyTorchResultProcessor.java:144)\n\tat org.elasticsearch.ml@8.13.4/org.elasticsearch.xpack.ml.inference.pytorch.process.PyTorchResultProcessor.process(PyTorchResultProcessor.java:137)\n\tat org.elasticsearch.ml@8.13.4/org.elasticsearch.xpack.ml.inference.deployment.DeploymentManager.lambda$startDeployment$2(DeploymentManager.java:180)\n\tat org.elasticsearch.server@8.13.4/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:917)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n"} 2024-05-16 14:48:00 {"@timestamp":"2024-05-16T12:48:00.161Z", "log.level": "INFO", "message":"[.ds-.logs-deprecation.elasticsearch-default-2024.05.16-000001] creating index, cause [initialize_data_stream], templates [.deprecation-indexing-template], shards [1]/[1]", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[f059c181c171][masterService#updateTask][T#8]","log.logger":"org.elasticsearch.cluster.metadata.MetadataCreateIndexService","elasticsearch.cluster.uuid":"eEqzvIFnQx6Lt08J4545wQ","elasticsearch.node.id":"-It6EisaTT2HVi05RqZGdA","elasticsearch.node.name":"f059c181c171","elasticsearch.cluster.name":"elasticsearch"} 2024-05-16 14:48:00 {"@timestamp":"2024-05-16T12:48:00.167Z", "log.level": "INFO", "message":"adding data stream [.logs-deprecation.elasticsearch-default] with write index [.ds-.logs-deprecation.elasticsearch-default-2024.05.16-000001], backing indices [], and aliases []", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[f059c181c171][masterService#updateTask][T#8]","log.logger":"org.elasticsearch.cluster.metadata.MetadataCreateDataStreamService","elasticsearch.cluster.uuid":"eEqzvIFnQx6Lt08J4545wQ","elasticsearch.node.id":"-It6EisaTT2HVi05RqZGdA","elasticsearch.node.name":"f059c181c171","elasticsearch.cluster.name":"elasticsearch"} ``` ---
**Author:** ![Tan\_H](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tan_h/32/135111_2.png) [@Tan\_H](https://discuss.elastic.co/u/Tan_H)\ **Post date:** [June 7, 2024, 8:27am UTC](https://discuss.elastic.co/t/elastic-search-trained-model-inference-not-working/359649/2 "2024-06-07T08:27:53Z")
I have the same issue ![Screenshot 2024-06-07 at 15.02.06](https://us1.discourse-cdn.com/elastic/original/3X/1/d/1db054d32e57deece6af814864556931fcc96d5a.png) ---
**Author:** ![carly.richmond](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carly.richmond/32/104935_2.png) [@carly.richmond](https://discuss.elastic.co/u/carly.richmond)\ **Post date:** [June 7, 2024, 8:31am UTC](https://discuss.elastic.co/t/elastic-search-trained-model-inference-not-working/359649/3 "2024-06-07T08:31:16Z")
@Tan_H and @likealam which version of Elastic are you using? Not sure if it helps but my colleague did share some tips on diagnosing ML issues in [this blog](https://www.elastic.co/search-labs/blog/ml-inference-pipelines-bottlenecks) that might be worth checking. ---
**Author:** ![Tan\_H](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tan_h/32/135111_2.png) [@Tan\_H](https://discuss.elastic.co/u/Tan_H)\ **Post date:** [June 7, 2024, 1:13pm UTC](https://discuss.elastic.co/t/elastic-search-trained-model-inference-not-working/359649/4 "2024-06-07T13:13:13Z")
Hi @carly.richmond , I running the latest version of the stack, from [GitHub - deviantony/docker-elk: The Elastic stack (ELK) powered by Docker and Compose.](https://github.com/deviantony/docker-elk) ---
**Author:** ![carly.richmond](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carly.richmond/32/104935_2.png) [@carly.richmond](https://discuss.elastic.co/u/carly.richmond)\ **Post date:** [June 8, 2024, 7:04am UTC](https://discuss.elastic.co/t/elastic-search-trained-model-inference-not-working/359649/5 "2024-06-08T07:04:54Z")
Thanks for confirming @Tan_H. I see you are using images published by a developer in the community. It's difficult to say if it's an issue with the container configuration. Did you make any changes to your local configuration as well? If so it would be useful to share your container configuration and we can take a look. If the troubleshooting blog isn't useful I would also recommend reaching out to the developer too. Hope that helps! ---
**Author:** ![mjavidan](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@mjavidan](https://discuss.elastic.co/u/mjavidan)\ **Post date:** [January 14, 2025, 8:45pm UTC](https://discuss.elastic.co/t/elastic-search-trained-model-inference-not-working/359649/6 "2025-01-14T20:45:17Z")
Hello, @Tan_H and @likealam did you manage to figure out what was the issue? I'm having the same problem.