i have an elastic instance where i try to set up a custom OpenAI connector to a machine with a locally hosted LLM to use within the AI Assistant. Since 8.17 it is possible to use an OpenAI compatible API connector to accomplish this.
Unfortunately, when i send a prompt from the AI Assistant, i get a rather generic errors in the AI Assistant Chat:
ActionsClientChatOpenAI: an error occurred while running the action - Status code: undefined. Message: Unexpected API Error: ERR_CANCELED - canceled.
ActionsClientChatOpenAI: an error occurred while running the action - Unexpected API Error: - Request was aborted.
I can see the incoming request from Kibana in the logs of lm-studio (the application which hosts the llm). After a short period of time however (after the request from Kibana comes in on the llm-host), a log in lm-studio appears which states "Client disconnected. Stopping generation ..." even before the llm is able to finish generating a response. When sending a prompt from the kibana host to the llm via curl directly to the API endpoint, i get a response back, thus i suspect the issue is related to Elastic/Kibana. The test of the connector succeeds.
Same problem. Kibana connecting to LLM and receiving responses stream. But Kibana AI assistant chat not showing the text. And after 60s wait - shows "Request was aborted." error. Changes in xpack.actions.responseTimeout parameter is not solve the problem
Sounds somewhat related. However Kibana AI Assistant Chat does not wait that long in my case. I get the error messages a couple of seconds later after i sent the chat message to the llm server. Kibana still somewhat behaves like it time-outs prematurely and setting parameter xpack.actions.responseTimeout did not yield any change in behavior. I also removed any other complexity already such as the reverse proxy which handles tls. Kibana directly speaks to the API of lm-studio hosting the model.
Update: Although i switched from lm-studio to the text-generation-webui (to somewhat try to address the API implications in the error message), Kibana still responds with the initial error message in the AI Assistant after five to ten seconds:
ActionsClientChatOpenAI: an error occurred while running the action - Status code: undefined. Message: Unexpected API Error: ERR_CANCELED - canceled
Kibana prematurely aborts the request (i.e. responds with the error and does not wait for the response) while the LLM is still generating its response on the other host.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.