Strange authentication error

Hello, I have been running the ELK stack 8.6.2 for a month now with no problems, but I had security disabled.

Today I attempted to enable security following this documentation:
https://www.elastic.co/guide/en/elasticsearch/reference/current/security-minimal-setup.html

I have the full ELK stack running on a server and I'm exposing Kibana through nginx. It's working (or was) just fine with 'xpack.security.enabled: false' on the elastic.yml. After following the above documentation in an attempt to setup minimal security, I have encountered an issue.

The issue is that I can't login to either the elastic user, or the kibana_system user through the dashboard. Now, I realize the kibana_system isn't supposed to be able to login on the dashboard and that's fine, but the elastic user can't either which means I can't access the dashboards at all.

The "strange" part about this error is that if I type in the elastic username and a random password, it immediately says password/username incorrect. However, if I type in the correct username and password, it appears to login and start loading for a moment, before I'm brought back to this screen:

I actually get to see this:
Screenshot from 2023-03-23 18-03-54
But then it just brings me back to the login error screen.

I apologize for any information I might not have given right off the bat, I'm fairly new to the ELK Stack. If there's any more information I can provide to help you help me, please do let me know.
I tried to find out how to read kibana logs so I could figure out what was happening but everything I found was through the dashboard, which I can't access at the moment.

Any help would be very much appreciated. Thank you.

Can you share the logs of both Kibana and Elasticsearch? Share the logs of the Elasticsearch instance configured in your kibana.yml.

Yes! I was actually about to edit to add logs.

var/log/kibana generates the following log entry when I try to authenticate:

{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.4.0"},"@timestamp":"2023-03-23T19:38:32.317+00:00","message":"Authentication attempt failed: {\"error\":{\"root_cause\":[{\"type\":\"security_exception\",\"reason\":\"unable to authenticate user [kibanaorwell] for REST request [/_security/_authenticate]\",\"header\":{\"WWW-Authenticate\":[\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\",\"ApiKey\"]}}],\"type\":\"security_exception\",\"reason\":\"unable to authenticate user [kibanaorwell] for REST request [/_security/_authenticate]\",\"header\":{\"WWW-Authenticate\":[\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\",\"ApiKey\"]}},\"status\":401}","log":{"level":"INFO","logger":"plugins.security.authentication"},"process":{"pid":5697},"trace":{"id":"58d53d53d4982eccbeb2a5dbeff5f5b9"},"transaction":{"id":"5b441aaed56ddd01"}}

This is strange because kibanaorwell is the username I have for Nginx before I had security enabled.

The following is the Elasticsearch log entry from when I tried to login:

[2023-03-23T19:31:54,179][INFO ][o.e.x.s.a.RealmsAuthenticator] [bigbrother] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]

I found this thread a couple of minutes ago and I think I might be suffering from a similar problem.

I apologize for not mentioning I had been using Nginx before I enabled security, I had no idea it could cause this sort of issue.

I am using Nginx to be able to access the Kibana dashboard from my ELK server's public IP address. This was a solution suggested to me at the time when I started with 7.x.
Is there no longer a need for this or do I still need to use Nginx to do it but have to change configurations?

Thank you for taking the time to reply. If there's anything else I can do or if I need to provide more logs just let me know.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.