We are facing issues with connecting to our local LLM from Elasticsearch cluster of 8.18.0 version. While we are able to connect the same model Llama 3.2 3B Instruct to the 8.15.1 Elasticsearch.
Getting the following error while testing the connector. The following error was found: an error occurred while running the action Details: Status code: undefined. Message: Unexpected API Error: ECONNREFUSED -
Error from security ai assistant ActionsClientChatOpenAI: an error occurred while running the action - Unexpected API Error: - Connection error.
Do you mind sharing more info on the connector configuration (URL, default model, and any advanced settings) you're using? It might help us track down what could be misaligned between the two versions.
Here are some changes between the two versions that might be tripping up the connector:
In Kibana 8.18, if you are pre-configuring the connector using kibana.yml, you have to set exposeConfig: true if you want your config to be exposed to the actions API - Read More
Have you checked out this guide on Connecting your own local LLM for version 8.18? I saw some changes between the two versions that might be causing the error.
For example, the 8.15 guide for connectors allowed endpoints to start with HTTP but in the 8.18 guide, your connector URL must start with HTTPS.
url:http://localhost:1234/v1/chat/completions (Got for lm studio logs)
OpenAi provider: OpenAi
Default model: gpt-4o
No extra settings are used neither in Kibana.yml also.
And lm studio by default creating Url with htpp not sure I can make it to run on https.
I guess I got what's the issue is but not sure about it.
so just confirm me.
As per the documentation if lm studio & Elastic instance or on the same host reverse proxy is not required, but in my case Elastic instance is on K8s server & lm studio is in local.
And if so is causing the issue need to know how to setup lm studio in the k8s.
I'm not sure which documentation you're referring to, or whether you're using this as part of the Elastic Security solution, but there is a guide that talks through setting up a reverse proxy here. Let us know if that's what you're looking for.
Can you give more information on what you mean by:
Its really is the issue with the supported models & Elasticsearch version.
It might also be worth checking the logs to see if there are more detailed errors coming through.
As @JD_Armada has stated from 8.18 your connector URL must start with https rather than http. It doesn't look like LM Studio supports https, so you would need to use a tunnelling service such as pinggy on the top and then use that https URL instead in your connector configuration.
Thank you for the response & suggestions, I tried pinggy with https url but still facing the same issue in security ai assistant. From connector when tested it shows successful sometimes but is absolutely failing to give response in ai assistant.
[2025-07-21T11:18:54.945+05:30][ERROR][plugins.elasticAssistant.defaultAssistantGraph] Error streaming graph: Unexpected API Error: - Connection error.: ActionsClientChatOpenAI: an error occurred while running the action - Unexpected API Error: - Connection error.
[2025-07-21T11:19:21.277+05:30][INFO ][plugins.fleet.fleet:unenroll-inactive-agents-task:1.0.0] [runTask()] started
[2025-07-21T11:19:21.306+05:30][INFO ][plugins.fleet.fleet:unenroll-inactive-agents-task:1.0.0] [UnenrollInactiveAgentsTask] runTask ended: success
[2025-07-21T11:19:54.214+05:30][ERROR][plugins.actions.gen-ai] Error: Unexpected API Error: - Connection error.
at OpenAIConnector.invokeAsyncIterator (C:\Users\AM27\Elasticsearch\kibana-8.18.0\node_modules\@kbn\stack-connectors-plugin\server\connector_types\openai\openai.js:278:13)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at Object.executor (C:\Users\AM27\Elasticsearch\kibana-8.18.0\node_modules\@kbn\actions-plugin\server\sub_action_framework\executor.js:70:18)
at C:\Users\AM27\Elasticsearch\kibana-8.18.0\node_modules\@kbn\actions-plugin\server\lib\action_executor.js:395:21
at ActionExecutor.execute (C:\Users\AM27\Elasticsearch\kibana-8.18.0\node_modules\@kbn\actions-plugin\server\lib\action_executor.js:76:12)
at C:\Users\AM27\Elasticsearch\kibana-8.18.0\node_modules\@kbn\langchain\server\language_models\chat_openai.js:116:28
at RetryOperation._fn (C:\Users\AM27\Elasticsearch\kibana-8.18.0\node_modules\p-retry\index.js:50:12)
[2025-07-21T11:19:54.215+05:30][WARN ][plugins.actions.gen-ai] action execution failure: .gen-ai:cdaed1cd-e417-4a27-b0fe-bd04192e96ad: my llama: an error occurred while running the action: Unexpected API Error: - Connection error.; retry: true
[2025-07-21T11:19:54.220+05:30][ERROR][plugins.elasticAssistant.defaultAssistantGraph] Error streaming graph: Unexpected API Error: - Connection error.: ActionsClientChatOpenAI: an error occurred while running the action - Unexpected API Error: - Connection error.
As for your question issue with the supported models & Elasticsearch version, when I connected with official Elastic help support team, they mentioned that Elasticsearch doesn't support local llm models fully only some are supported & even so there are issues working with that models. Large language model performance matrix | Elastic Docs
as per this documentation llama is supported & i am using that only but facing the connection issues. Elastic Support Hub
Thanks @Abhi_Abhishek. Do you have an open case with support? They may be able to advise further, but diving into the LLM performance matrix Llama 3.2 is stated as having Good or Poor support. Have you tried using those listed as having Excellent support such as Mistral‑Small‑3.2‑24B‑Instruct‑2506 or Mistral-Small-3.1-24B-Instruct-2503?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.