# AI Assistant with local LLM

**URL:** <https://discuss.elastic.co/t/ai-assistant-with-local-llm/383091>\
**Category:** Elasticsearch\
**Tags:** elastic-ai-assistant\
**Created:** [October 30, 2025, 9:33am UTC](https://discuss.elastic.co/t/ai-assistant-with-local-llm/383091 "2025-10-30T09:33:46Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![arcsons](https://avatars.discourse-cdn.com/v4/letter/a/5fc32e/32.png) [@arcsons](https://discuss.elastic.co/u/arcsons)\
**Post date:** [October 30, 2025, 9:33am UTC](https://discuss.elastic.co/t/ai-assistant-with-local-llm/383091/1 "2025-10-30T09:33:46Z")

</div>

Hi,

I’m trying to setup an AI connector for my local LLM.  
I have ngnix as reverse proxy with an API key.

$ curl -s [http://x.x.x.x:8080/api/chat](http://x.x.x.x:8080/api/chat) -H "Authorization: $OKEY" -H "Content-Type: application/json" -d '{  
"model": "llama3.1:8b-instruct-q4\_K\_M",  
"messages": [  
{"role":"system","content":"You are helpful."},  
{"role":"user","content":"Say hello in one short sentence."}  
],  
"stream": false  
}'  
{"model":"llama3.1:8b-instruct-q4\_K\_M","created\_at":"2025-10-30T09:19:56.546722543Z","message":{"role":"assistant","content":"Hello!"},"done":true,"done\_reason":"stop","total\_duration":18342984383,"load\_duration":4902469261,"prompt\_eval\_count":26,"prompt\_eval\_duration":12297094129,"eval\_count":3,"eval\_duration":1134860182} $

Now in Kibana.

My connector settings looks like:

 ![api2](https://us1.discourse-cdn.com/elastic/original/3X/e/1/e19a500332a1c75ee3c3e5152d5b5a4cbdd090c5.png)

And my test error:

 ![api1](https://us1.discourse-cdn.com/elastic/original/3X/c/9/c956e263cc643a86b1528ce671bfe64270cae04e.png)

Any idea what I have configured wrong?

Thanks for help!

---

<div class="post-metadata">

**Author:** ![carly.richmond](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carly.richmond/32/104935_2.png) [@carly.richmond](https://discuss.elastic.co/u/carly.richmond)\
**Post date:** [October 30, 2025, 10:11am UTC](https://discuss.elastic.co/t/ai-assistant-with-local-llm/383091/2 "2025-10-30T10:11:42Z")

</div>

It looks like an authorization error. Have you checked the key is correct and that you have access to the specified model using that key? If not you could try regenerating the key.

---

<div class="post-metadata">

**Author:** ![arcsons](https://avatars.discourse-cdn.com/v4/letter/a/5fc32e/32.png) [@arcsons](https://discuss.elastic.co/u/arcsons)\
**Post date:** [October 30, 2025, 11:55am UTC](https://discuss.elastic.co/t/ai-assistant-with-local-llm/383091/3 "2025-10-30T11:55:28Z")

</div>

The key works fine with curl. The modell I use is the same as well.

curl -s -H "Authorization: Bearer sk\_local\_423322d2003e436d78f288ce3f3af4057ccea2e225454da5" [http://x.x.x.x:8080/api/tags](http://x.x.x.x:8080/api/tags) | jq .  
{  
"models": [  
{  
"name": "llama3.1:8b-instruct-q4\_K\_M",  
"model": "llama3.1:8b-instruct-q4\_K\_M",  
"modified\_at": "2025-10-30T09:05:34.577289698+01:00",  
"size": 4920753328,  
"digest": "46e0c10c039e019119339687c3c1757cc81b9da49709a3b3924863ba87ca666e",  
"details": {  
"parent\_model": "",  
"format": "gguf",  
"family": "llama",  
"families": [  
"llama"  
],  
"parameter\_size": "8.0B",  
"quantization\_level": "Q4\_K\_M"  
}  
}  
]  
}

What should i put in the URL for the settings?

---

<div class="post-metadata">

**Author:** ![arcsons](https://avatars.discourse-cdn.com/v4/letter/a/5fc32e/32.png) [@arcsons](https://discuss.elastic.co/u/arcsons)\
**Post date:** [November 5, 2025, 10:29pm UTC](https://discuss.elastic.co/t/ai-assistant-with-local-llm/383091/4 "2025-11-05T22:29:53Z")

</div>

Please anyone? 🙃

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [November 5, 2025, 10:52pm UTC](https://discuss.elastic.co/t/ai-assistant-with-local-llm/383091/5 "2025-11-05T22:52:25Z")

</div>

> [@arcsons](#):
>
> I have ngnix as reverse proxy with an API key.

Have you checked your NGINX logs? How did you configure it?

---

<div class="post-metadata">

**Author:** ![arcsons](https://avatars.discourse-cdn.com/v4/letter/a/5fc32e/32.png) [@arcsons](https://discuss.elastic.co/u/arcsons)\
**Post date:** [November 6, 2025, 3:03pm UTC](https://discuss.elastic.co/t/ai-assistant-with-local-llm/383091/6 "2025-11-06T15:03:43Z")

</div>

Hey Leonard, thanks for replay.

Ngnix config:

/var/log/nginx# cat /etc/nginx/sites-available/ollama

```auto

/var/log/nginx# cat /etc/nginx/sites-available/ollama
server {
# Change to 80/443 if desired (443 requires TLS config below)
listen 8080;
listen \[::\]:8080;

`
# Optional: restrict who can reach this (uncomment and set your LAN/CIDR)
# allow 192.168.1.0/24;
# deny all;

# Simple static Bearer-token check
set $expected_key "Bearer sk_local_423322d2003e436d78f288ce3f3af4057ccea2e225454da5";

# If header doesn't match, return 401
if ($http_authorization != $expected_key) {
    return 401;
}

# Optional: add a WWW-Authenticate header for clarity
add_header WWW-Authenticate 'Bearer realm="ollama"';

location / {
    proxy_pass http://127.0.0.1:11434;
    proxy_http_version 1.1;

    # Pass through headers
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;

    # For streaming
    proxy_buffering off;
}

}

```

```auto
==> access.log <==
10.10.0.22 - - \[06/Nov/2025:15:43:46 +0100\] "POST /v1/chat HTTP/1.1" 401 188 "-" "axios/1.12.1"

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [November 6, 2025, 4:01pm UTC](https://discuss.elastic.co/t/ai-assistant-with-local-llm/383091/7 "2025-11-06T16:01:11Z")

</div>

Curious why urls are different

config ... `/v1/chat`

and when thne curl

`curl` ,... `/api/chat`

---

<div class="post-metadata">

**Author:** ![arcsons](https://avatars.discourse-cdn.com/v4/letter/a/5fc32e/32.png) [@arcsons](https://discuss.elastic.co/u/arcsons)\
**Post date:** [November 6, 2025, 8:55pm UTC](https://discuss.elastic.co/t/ai-assistant-with-local-llm/383091/8 "2025-11-06T20:55:22Z")

</div>

Testing around but nothing seems to work. I have no clue what URL I should use.

> $ curl -s [http://localhost:8080/v1/chat/completions](http://localhost:8080/v1/chat/completions)  
> "Content-Type: application/json"  
> -H\> -H "Content-Type: application/json" \
> 
> > -H "Authorization: Bearer sk\_local\_423322d2003e436d78f288ce3f3af4057ccea2e225454da5"  
> > -d '{  
> > "model": "llama3.1:8b-instruct-q4\_K\_M",  
> > "messages": [  
> > {"role": "user", "content": "Hello, how are you today?"}  
> > ]  
> > }'

Works fine. But When I use that URL in Kibana connector it doesn't work.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [November 6, 2025, 9:21pm UTC](https://discuss.elastic.co/t/ai-assistant-with-local-llm/383091/9 "2025-11-06T21:21:25Z")

</div>

Assuming you are following this

> **[Connect to your own local LLM | Elastic Docs](https://www.elastic.co/docs/solutions/security/ai/connect-to-own-local-llm)**
>
> This page provides instructions for setting up a connector to a large language model (LLM) of your choice using LM Studio. This allows you to use your...

The URL should be

> 1. Under **URL** , enter the domain name specified in your Nginx configuration file, followed by `/v1/chat/completions`.