I just deployed Shield + Shield plugin for Kibana. All is working but one major issue is happening ::
The Shield login page will accept any username/password combination and allows into Kibana to see everything!!!!
What is happening?
Did you do any special configuration? If you can share your configuration then that would also be helpful including versions of Shield (both elasticsearch and kibana side), Elasticsearch and Kibana. Do you have anonymous access enabled?
Hi Jay,
Which configuration would you like me to post? I am using latest and greatest (ES 2.2, Kibana 4.4, Shield latest). I don't think I have anonymous access enabled, but where can I check that?
I did comment out auth_basic "Restricted Area"; and auth_basic_user_file lines in nginx-kibana.conf.....because it was asking me for authentication before I could access the Shield login page
Any roles that you have modified would be the first thing I'd look at.
In your elasticsearch.yml, you would have shield.authc.anonymous.roles set if anonymous access is enabled.
I am not familiar with this configuration. I am guessing your kibana instance is behind a reverse proxy?
Also, are you logging out before trying to log in as a different user?
-
I have not modified anything inside of the roles.yml file located at /etc/elasticsearch/shield/roles.yml
-
I do not have that 'shield.authc.anonymous.roles' set so anonymous access is not enabled it appears
-
Our Kibana instances uses the NGINX web server to act as a reverse proxy. Logging out/in does not matter, as I can visit any machine (even machines that have never tried to access Kibana) and authenticate with any username/password combination.
Hi Michael,
Can you confirm that you've installed Shield on all nodes in your cluster (rebooting each node after installing the license and shield plugins) and that you have the same users, roles and user_role files on all nodes?
Can you try accessing Kibana directly, removing NGINX from the scenario? This will eliminate variables we can't control, and make it easier for us to help.
If you go directly to Kibana, you should see a login page (not the browser pop-up). Once you authenticate, you will be brought to the Kibana Application, and you will see a logout button in the top-right. If you want to log in as another user, you will need to log out first.
Thanks,
Steve
A few other questions in addition to Steve's, if you hit elasticsearch directly are you prompted for credentials? What type of license are you using? You can check license status from elasticsearch with curl -XGET -u <admin_user> 'http://<host>:<port>/_license'
Okay, since my last post, now I am trying to use Active Directory to login. I am able to get to the login screen, and entering invalid credentials does show a message saying credentials are invalid. When I type in my domain credentials that is apart of the OU I configured to allow power_user access to, it logs me in, but it brings me to the Server Status page, which is in the red, with a message ::
plugin:elasticsearch [security_exception] action [cluster:monitor/nodes/info] is unauthorized for user [ELKAD]
ELKAD is a user I created in Active Directory. I gave that user Domain Admin rights, and I also created the user via ./esusers and assigned role of admin for that user. I added the username and password to kibana.yml as well, (elasticsearch.username and elasticsearch.password). Restarted ES and Kibana.
If this user is assigned to the admin role, why am I getting that message above?
So authenticating with AD credentials seems to work, because typing in the wrong credentials do not. I assume there is further authentication that takes place beyond that, such as from Kibana to ES or?
As of right now, I have reverted what I configured, I deleted the user in AD that I created for ELK to use because I am not sure if it is even needed. I also deleted the user from ES (./esusers userdel ELKAD)
Hopefully you guys can provide some further insight as to what needs to be done beyond the point of being able to authenticate using AD credentials. Right now after I do that, I get to the server status page and it says plugin:elasticsearch Authentication Exception
Hi Michael,
I am a little bit lost with where you are at right now. Do you still have the issue where anything in the login page allows access to elasticsearch? If so, I think we should focus on that aspect first before moving to active directory.
If not, then can you detail the configuration you used for active directory a bit more including sharing your role mapping? Also, active directory users do not need to be created in elasticsearch (ie using the esusers tool).
-Jay
Here is where I am at ::
- I visit the ELK URL and am presented with a login screen (Shield login)
- I enter wrong AD credentials to verify it is contacting the domain controller and authenticating correctly, this is proven true due to the fact it rejects wrong credentials but accepts correct ones
- After I enter correct credentials, it brings me to the 'Server Status' page where all services are in 'Ready' status EXCEPT for plugin:elasticsearch which says 'Authentication Exception' and plugin:marvel 'Waiting for Elasticsearch'
Here is what is in my elasticsearch.yml ::
shield.authc.realms:
active_directory:
type: active_directory
domain_name: mydomain.com
unmapped_groups_as_roles: true
url: ldap://my-dc.mydomain.com
I have the elasticsearch.username and elasticsearch.password commented out in my kibana.yml file, currently. I am not sure what to put for these as I am integrating with AD. Might be a problem, might not be, I am not sure at this time.
In role-mapping.yml, I do have the correct DistinguishedName to the OU the users are in, in AD, for authentication.
I can verify the shield plugin as well as license are installed on all four of my nodes.
That is a problem. Kibana needs a user with the kibana4_server role. I recommend creating a esusers user for it. Step 1 on this page: Using Kibana with Shield | Shield [2.4] | Elastic
You would need to update your realm configuration:
shield.authc.realms:
esusers:
type: esusers
order: 0
active_directory:
type: active_directory
order: 1
domain_name: mydomain.com
url: ldap://my-dc.mydomain.com
unmapped_groups_as_roles: true
Once you have Kibana talking to elasticsearch, I suggest looking at role mapping.
You are attempting to use a OU, which is not supported. The role mapping works on the DN of the user and the group DNs that they belong to. Please refer to: https://www.elastic.co/guide/en/shield/current/active-directory.html#assigning-roles-ad
If you still have issues, I recommend setting the log level of elasticsearch to DEBUG or set shield.authc: DEBUG in the logger section of your logging.yml. This should print out the group DNs that were retrieved from AD and what was mapped.
Jay,
My mistake, I am not trying to use an OU, I am using a group. I know it works because I was able to make it to the server status page with my domain credentials.
So, I uncommented out the elasticsearch.username and elasticsearch.password lines, and made sure it has the kibana4-server account with the correct password, which does exist in esusers.
I also added those extra lines to the realm as you suggested.
After all of that, I restarted Kibana and ES, the same thing is happening. I get to the server status page just fine with my domain credentials, but plugin:elasticsearch still says 'Authentication Exception'
Partial output of 'sudo docker logs elasticsearch' ::
authentication failed for user [kibana4-server]: unable to authenticate user [kibana4-server] to active directory domain [mydomain.com]
cause: com.unboundid.ldap.sdk.LDAPException: 80090308: LdapErr: DSID-0C0903CF, comment: AcceptSecurityContext error, data 52e, v2580
Hmm that indicates kibana4-server is failing authentication via esusers. Did you add the order lines? Can you issue curl localhost:9200 -u kibana4-server and see what happens?
Here is the output of that ::
[elk@elk01 elk-conf]$ sudo curl localhost:9200 -u kibana4-server
Enter host password for user 'kibana4-server': I ENTERED PASSWORD FOR kibana4-server
{"error":{"root_cause":[{"type":"security_exception","reason":"unable to authenticate user [kibana4-server] for REST request [/]","header":{"WWW-Authenticate":"Basic realm="shield""}}],"type":"security_exception","reason":"unable to authenticate user [kibana4-server] for REST request [/]","header":{"WWW-Authenticate":"Basic realm="shield""}},"status":401}
It does not appear that the esusers realm is working properly. If you look in $ES_HOME/config/shield/users do you see the kibana4-server user?
I went to /etc/elasticsearch/shield, did 'vi users' and saw the kibana4-server user inside with a hashed password.
I cannot find 'users' in any other place besides the above location.
Just for tests, I even tried the curl using other users inside 'users' but all resulted the same.
Are you using a RPM or DEB install? Is there anything in your elasticsearch logs relating to users/roles when you start up?
I am using docker containers. To spin up ES and Kibana I just issue a 'docker run' and specify which version I want to run.
Logs filled with ::
[2016-03-24 18:52:43,106][WARN ][shield.authc.activedirectory] [Training1 - Wolverine] authentication failed for user [kibana4-server]: unable to authenticate user [kibana4-server] to active directory domain [mydomain.com]
cause: com.unboundid.ldap.sdk.LDAPException: 80090308: LdapErr: DSID-0C0903CF, comment: AcceptSecurityContext error, data 52e, v2580
[2016-03-24 18:52:45,654][WARN ][shield.authc.activedirectory] [Training1 - Wolverine] authentication failed for user [kibana4-server]: unable to authenticate user [kibana4-server] to active directory domain [mydomain.com]
cause: com.unboundid.ldap.sdk.LDAPException: 80090308: LdapErr: DSID-0C0903CF, comment: AcceptSecurityContext error, data 52e, v2580
[2016-03-24 18:52:48,199][WARN ][shield.authc.activedirectory] [Training1 - Wolverine] authentication failed for user [kibana4-server]: unable to authenticate user [kibana4-server] to active directory domain [mydomain.com]
cause: com.unboundid.ldap.sdk.LDAPException: 80090308: LdapErr: DSID-0C0903CF, comment: AcceptSecurityContext error, data 52e, v2580
[2016-03-24 18:52:50,748][WARN ][shield.authc.activedirectory] [Training1 - Wolverine] authentication failed for user [kibana4-server]: unable to authenticate user [kibana4-server] to active directory domain [mydomain.com]
cause: com.unboundid.ldap.sdk.LDAPException: 80090308: LdapErr: DSID-0C0903CF, comment: AcceptSecurityContext error, data 52e, v2580
[2016-03-24 18:52:53,292][WARN ][shield.authc.activedirectory] [Training1 - Wolverine] authentication failed for user [kibana4-server]: unable to authenticate user [kibana4-server] to active directory domain [mydomain.com]
cause: com.unboundid.ldap.sdk.LDAPException: 80090308: LdapErr: DSID-0C0903CF, comment: AcceptSecurityContext error, data 52e, v2580
[2016-03-24 18:52:55,836][WARN ][shield.authc.activedirectory] [Training1 - Wolverine] authentication failed for user [kibana4-server]: unable to authenticate user [kibana4-server] to active directory domain [mydomain.com]
cause: com.unboundid.ldap.sdk.LDAPException: 80090308: LdapErr: DSID-0C0903CF, comment: AcceptSecurityContext error, data 52e, v2580