Elasticsearch Kibana 9.4.0 Basic authentication returns 401 Unauthorized

Elastic Stack Kibana
21

FYI I added

logging.loggers:
  - name: security
    level: trace
  - name: plugins.security
    level: trace

to my kibana.yml and get some more diagnostics.

This includes somewhat confusing message like this set for a successful (httpCode 200) curl call to /api/features with the `-H 'Authorization: Basic xxxxx' header:

DEBUG  plugins.security.basic.basic                                         2026-05-19T16:47:04.147+02:00  Trying to authenticate user request to /api/features.
DEBUG  plugins.security.basic.basic                                         2026-05-19T16:47:04.147+02:00  Cannot authenticate requests with `Authorization` header.
DEBUG  plugins.security.http                                                2026-05-19T16:47:04.147+02:00  Trying to authenticate user request to /api/features.
DEBUG  plugins.security.http                                                2026-05-19T16:47:04.149+02:00  "Request to /api/features has been authenticated via authorization header with ""Basic"" scheme."
22

I did a fresh single node installation of version 9.3.0 and version 9.4.0 with minimal configuration

9.3.0 no problem

9.4.0 -> authentication error for Kibana

Installation files :

elasticsearch-9.3.0-linux-x86_64.tar.gz
kibana-9.3.0-linux-x86_64.tar.gz

elasticsearch-9.4.0-linux-x86_64.tar.gz
kibana-9.4.0-linux-x86_64.tar.gz

Users :

  • testuser
./elasticsearch-users useradd testuser -p testpassword -r superuser

echo -n 'testuser:testpassword' | base64
dGVzdHVzZXI6dGVzdHBhc3N3b3Jk
  • accessuser ( used for connection between kibana and elasticsearch )
./elasticsearch-users useradd accessuser -p accesspassword -r kibana_system

echo -n 'accessuser:accesspassword' | base64
YWNjZXNzdXNlcjphY2Nlc3NwYXNzd29yZA==

Elasticsearch configuration :

elasticsearch.yml

cluster.name: my-application-cluster
node.name: node-1
network.host: localhost
discovery.type: single-node

xpack.security.enabled: false
xpack.security.enrollment.enabled: false
xpack.security.http.ssl.enabled: false
xpack.security.transport.ssl.enabled: false

users :

testuser:$2a$10$Ovv7PFp9yIqNUxjA0saJdeYhbvRnG1WQjtq5.RQ9SD0LxenDk/Ppm
accessuser:$2a$10$qYcpf4o/syrmcsKLrZsk/uBlhtML7/OQPcKLr2Ynyo0N2uqqQZldq

users_roles :

kibana_system:accessuser
superuser:testuser

Kibana configuration :

kibana.yml :

server.port: 8100
server.host: "127.0.0.1"
server.ssl.enabled: false

elasticsearch.hosts: "http://127.0.0.1:9200"
elasticsearch.username: accessuser
elasticsearch.password: accesspassword

Curl cmd used to test with 'testuser' :

curl -v -H 'Authorization: Basic dGVzdHVzZXI6dGVzdHBhc3N3b3Jk' http://127.0.0.1:9200/_cat/indices

curl -v -H 'Authorization: Basic dGVzdHVzZXI6dGVzdHBhc3N3b3Jk' http://127.0.0.1:8100/api/spaces/space

curl -v -H 'Authorization: Basic dGVzdHVzZXI6dGVzdHBhc3N3b3Jk' http://127.0.0.1:8100/app/id/any*

green open .internal.alerts-transform.health.alerts-default-000001            NUkXE6vMTbaCEANgUQxQPQ 1 0 0 0 227b 227b 227b
green open .internal.alerts-observability.logs.alerts-default-000001          Vw0qWTh7S72oJP9-5xqeQg 1 0 0 0 227b 227b 227b
green open .internal.alerts-observability.uptime.alerts-default-000001        efx5KZIGRvmrilwdefxocg 1 0 0 0 227b 227b 227b
green open .internal.alerts-ml.anomaly-detection.alerts-default-000001        2ptaNeEoQqKChdRP7hdX7Q 1 0 0 0 227b 227b 227b
green open .internal.alerts-observability.slo.alerts-default-000001           cdkCrLY3TDWxcXkVThC1xw 1 0 0 0 227b 227b 227b
green open .internal.alerts-observability.apm.alerts-default-000001           DnfCvTRIRpqhkDkJFa-mQw 1 0 0 0 227b 227b 227b
green open .internal.alerts-default.alerts-default-000001                     2RLWLTDrQye5t-exyGCsGw 1 0 0 0 227b 227b 227b
green open .internal.alerts-streams.alerts-default-000001                     9dc4P_q5TfOUwdponGkxTA 1 0 0 0 227b 227b 227b
green open .internal.alerts-security.attack.discovery.alerts-default-000001   HNNsk85oQXGLT6nMsN7C1Q 1 0 0 0 227b 227b 227b
green open .internal.alerts-observability.metrics.alerts-default-000001       gdbgaMm4QUqgh7o9kroSkw 1 0 0 0 227b 227b 227b
green open .internal.alerts-observability.threshold.alerts-default-000001     U5trAxppTVW57UxSIuNPfg 1 0 0 0 227b 227b 227b
green open .internal.alerts-ml.anomaly-detection-health.alerts-default-000001 LNRgcuSLS5SzUe2aCQNa7A 1 0 0 0 227b 227b 227b
green open .internal.alerts-security.alerts-default-000001                    9q3_w46OSw2vNTlpTmqNCg 1 0 0 0 227b 227b 227b
green open .internal.alerts-dataset.quality.alerts-default-000001             j_eM8jHGRlWIF3UEIXDqIQ 1 0 0 0 227b 227b 227b
green open .internal.alerts-stack.alerts-default-000001                       Zz1L5bgHTpawesknNVOQsA 1 0 0 0 227b 227b 227b

green open .internal.alerts-security.alerts-default-000001 x1rd8iDKRtqu-zEASRP4jw 1 0 0 0 249b 249b 249b

[2026-05-20T14:47:39.185+02:00][DEBUG][plugins.security.authentication] Authentication is not required, as security features are disabled in Elasticsearch.
[2026-05-20T14:47:43.729+02:00][DEBUG][plugins.security.authentication] Authentication is not required, as security features are disabled in Elasticsearch.
[2026-05-20T14:47:43.767+02:00][DEBUG][plugins.security.user-profile] Request to get current user profile is authenticated via Basic credentials.
[2026-05-20T14:47:43.768+02:00][DEBUG][plugins.security.user-profile] Activating user profile via password grant.
[2026-05-20T14:47:43.781+02:00][ERROR][plugins.security.user-profile] Failed to activate user profile: {"error":"no handler found for uri [/_security/profile/_activate] and method [POST]"}.
[2026-05-20T14:47:43.782+02:00][DEBUG][plugins.security.user-profile] Failed to activate profile via basic credentials: {"error":"no handler found for uri [/_security/profile/_activate] and method [POST]"}
[2026-05-20T14:47:43.783+02:00][ERROR][http] 500 Server Error
23

Hi @GeertVerbeurgt
Thank you for the detail
But Sorry Which user / role are you using for each curl ... it is not clear
I am looking.... trying to reproduce

24

@stephenb

You can see it in my previous post .

The base64-encoded string is the one generated for 'testuser'

25

great @GeertVerbeurgt , as I was able to completely reproduce with steps you gave, on my spare linux system

Several differences to how I tested before, you have disabled security completely here, I was using Mac before, ... but I see identical to you:

[2026-05-20T16:44:48.106+02:00][DEBUG][plugins.security.authentication] Authentication is not required, as security features are disabled in Elasticsearch.
[2026-05-20T16:44:48.134+02:00][DEBUG][plugins.security.authentication] Authentication is not required, as security features are disabled in Elasticsearch.
[2026-05-20T16:44:48.147+02:00][DEBUG][plugins.security.user-profile] Request to get current user profile is authenticated via Basic credentials.
[2026-05-20T16:44:48.147+02:00][DEBUG][plugins.security.user-profile] Activating user profile via password grant.
[2026-05-20T16:44:48.156+02:00][ERROR][plugins.security.user-profile] Failed to activate user profile: {"error":"no handler found for uri [/_security/profile/_activate] and method [POST]"}.
[2026-05-20T16:44:48.156+02:00][DEBUG][plugins.security.user-profile] Failed to activate profile via basic credentials: {"error":"no handler found for uri [/_security/profile/_activate] and method [POST]"}
[2026-05-20T16:44:48.157+02:00][ERROR][http] 500 Server Error

Minor adendum - if you make this call

as kibana is starting up, there's is a brief window where you will get the 200 back.

26

@RainTown @stephenb

I got this mail for my previous post :

Our automated spam filter, Akismet, has temporarily hidden your post in Elasticsearch Kibana 9.4.0 Basic authentication returns 401 Unauthorized for review. 

A staff member will review your post soon, and it should appear shortly. 

We apologize for the inconvenience .
27

Hi @GeertVerbeurgt

I reviewed your post, all good, reenabled your post, cause = lots of URLs triggers the :robot:

AND I reproduced your issue.

I do see accessing those URLs via curl worked on 9.3.0 with the kibana_system role
I do see accessing those URLs via curl it did NOT work in 9.4.1with the kibana_system role

I did not understand you were using the kibana_system role for those curl s. I am still not sure if your intended use is to use curl for what you are trying to accomplish and why.

I will try to check internally if I can, no promise...
But I suspect:
A) That should not have ever worked for those URLs in the first place
B) and / or may have been a security fix of some sort to tighten security
C) All of the above

The kibana_system role is very specific, it not a role to access Kibana Assets / UI / Apps it is purely a role to allow the kibana server access elasticsearch as perform necessary functions. Personally, I am surprised that ever worked.

That role is not intended to be used externally in any way or for any other use. User beware.

kibana_system

Grants access necessary for the Kibana system user to read from and write to the Kibana indices, manage index templates and tokens, and check the availability of the Elasticsearch cluster. It also permits activating, searching, and retrieving user profiles, as well as updating user profile data for the kibana-* namespace. This role grants read access to the .monitoring-* indices and read and write access to the .reporting-* indices. For more information, see Configuring Security in Kibana.

I suspect you will need to create a different user / role combination with the proper permissions if you want to use curl in the way you are showing to access UI elements etc.

Apologies if I am still missing the point

28

Try with the other user created, with superuser role, created via

$ ./elasticsearch-users useradd testuser -p testpassword -r superuser

$ echo -n 'testuser:testpassword' | base64
dGVzdHVzZXI6dGVzdHBhc3N3b3Jk

You will likely find same diff from 9.3.0 (200 returned) to 9.4.0/9.4.1 (401, with kibana logging 500 Server Error)

29

"Likely" ? Sorry not following.

I just tested and both the elastic and testuser users both work for the use case as I would expect.

./elasticsearch-users useradd testuser -p mypassword -r superuser

curl -k -H "Authorization: Basic <correctcreds>" "http://localhost:5601/app/id/any*"

Just tested on 9.4.1 these both work because they a "user" roles not reserved / special roles

At this point I am totally confused

I just created a fully secured 9.4.1

I added

./elasticsearch-users useradd testuser -p mypassword -r superuser

 echo -n testuser:mypassword | base64                                                              
dGVzdHVzZXI6bXlwYXNzd29yZA==

curl -k -H "Authorization: Basic dGVzdHVzZXI6bXlwYXNzd29yZA==" "http://localhost:5601/app/id/any*" > 9_4_1.txt

Returns correct content

head 9_4_1.txt 
<!DOCTYPE html><html lang="en"><head><meta charSet="utf-8"/><meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"/><meta name="viewport" content="width=device-width"/><title>Elastic</title><style>
        
        @font-face {
          font-family: 'Elastic UI Numeric';
          font-style: normal;
          font-weight: 100 900;
          font-display: swap;
          src: url('/6b92ccb460a7/ui/fonts/elastic_ui_numeric/ElasticUINumeric-Variable.woff2') format('woff2');
          unicode-range: U+20, U+24-25, U+28-29, U+2B-2F, U+30-3A, U+A0, U+202F, U+2212;
        }
...

To me everything is working as expected....

  • Special roles like kibana_system not intended for curl use do not work for the use case provided.
  • User roles do work with curl for the use case provided
  • It does look like something changed for the special / reserved roles between 9.3.0 and 9.4.0 I took a quick look in the repo I did not see anything specific but it is a big set of repos
30

"Likely" meant only I'm not sure what you will see, so I don't want make assumptions :slight_smile:

You referenced the users with the kibana_system role, and thats acknowledged. But I had tested with the user created with superuser role exactly as @GeertVerbeurgt had written and done.

$ ./elasticsearch-users useradd testuser -p testpassword -r superuser
$ echo -n 'testuser:testpassword' | base64
dGVzdHVzZXI6dGVzdHBhc3N3b3Jk

And with that user, created exactly that way, using that Authorization header, I also see the error with 9.4.0 that I shared before, with these logs:

The way @GeertVerbeurgt had set things up here is a bit unconventional, with security pretty much entirely disabled, and using the user named accessuser , with only the kibana_system role as the "connecting" user, that is defined in kibana.yml

though why bother if security is disabled at elasticsearch end!

Then he/I tested the curl commands to kibana with the creds of the other user, username=testuser, who has superuser role.

In all this, easy to lose sight of the initial issue, which is that he has apache in front of Kibana and had added the apache config he gave:

and the curl commands were just an easier way to try to debug that client <-> apache <-> kibana exchange and see where the "401" was coming from.

I think this is a key difference. Because that is exactly what I had tested with already, and had been unable to reproduce, albeit with 9.4.0.

31

Thanks @RainTown I will let you take it from here... I am not really here debug proxy issue.

I am answering from what these roles are expected to do and not do.

I did find an Epic on tightening / auditing the kibana_system role / user. I suspect it was tightened as part of that.

I just set up 9.4.1 and that works above... I just showed that... above... I can provide the Docker compose all commands.. I am very confident that works (no proxy, no other components just Elastic and Kibana)

I showed all the steps, the curl and the output so either there is a diference between 9.4.0 and 9.4.1 or one of us is doing something different / wrong.

the curl with role superuser should absolutely work.

I will let you two work it out from here ...

I will test 9.4.0 real quick to see if I see something different and report back

  • There are role expected to be used with user interaction
  • There are reserved / special roles that are not expected to be used by users.
32

Absolutely. And I can make it not work, actually give an internal error, which is not ideal.

But that's only with Kibana and Elasticsearch with no security at all, sending calls with a header corresponding to auth for a superuser-role user.

Daft analogy, but the keys not working on a house with only wide open doors is a somewhat unusual issue.

I dont think 9.4.0 vs 9.4.1 will be any different.

33

To respond back to you @GeertVerbeurgt , the setup which I agree reproduces an issue is ... a bit unusual.

If I merely enabled security in elasticsearch in 9.4.0, the error goes away

xpack.security.enabled: true

as seen here:

curl -v -s -H 'Authorization: Basic dGVzdHVzZXI6dGVzdHBhc3N3b3Jk' "http://127.0.0.1:38100/app/id/any*" > out.tx
t
*   Trying 127.0.0.1:38100...
* Connected to 127.0.0.1 (127.0.0.1) port 38100
> GET /app/id/any* HTTP/1.1
> Host: 127.0.0.1:38100
> User-Agent: curl/8.5.0
> Accept: */*
> Authorization: Basic dGVzdHVzZXI6dGVzdHBhc3N3b3Jk
>
< HTTP/1.1 200 OK

Is that same for you?

Your 2 test setups had no security at all. That can't be your "real" setup, at time this thread started, though I appreciate you were trying to reduce to simplest reproducible case.

In your original report, you had this snippet:

RequestHeader set Authorization "expr=Basic {base64-encoded string}"
ProxyPass ``https://127.0.0.1:8100/
ProxyPassReverse ``https://127.0.0.1:8100/

What were the specific roles for the user corresponding to those base64 encoded creds?

And what are the roles for the user you are using in your kibana.yml

It would be better to try duplicate that setup IMHO.

34

9.4.0 Same Behavior

@RainTown I think you are getting closer I think something in the "simplification" is getting lost

35

@RainTown @stephenb

I have found the problem.

It was related to a special character ';' in the password of the login user, generated by a tool.

This was working until version 9.3.0. However, in the latest version, this causes problems during authentication.

Thank you so much for your help and support. I have already gained a few new insights into the elk stack.