Hi Team,
Just now started with shield, Active Directory configurations and AD user authentication got failed. In the trace log i found .security index not found error, PFA snapshot.
I created an esuser and it getting authenticated, not sure how to create .security index. Can you please suggest.
Thanks & Regards
Can you share the log message? In any case this is probably normal; the security index will not be created until you create a user or role via the API and you should not attempt to create it manually.
Hi Jay,
PFB logs, let me know if you need more of it.
Also attaching configurations i have done so far. gp-ops-es-admin is a AD group and i am expecting it will pull all users associated with that group.
*test1 is an Active directory user and admin is an esuser
*remove duplicate lines from the log due to bytes restrictions
Log:
[2016-08-24 16:08:47,232][INFO ][gateway ] [es-node-1] recovered [72] indices into cluster_state
[2016-08-24 16:09:08,323][INFO ][cluster.routing.allocation] [es-node-1] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[syslog-2016.08.19][0], [.kibana][0], [syslog-2016.08.19][0], [syslog-2016.08.19][3]] ...]).
[2016-08-24 16:09:13,622][DEBUG][shield.authc.esnative ] [es-node-1] user not found in cache, proceeding with normal authentication
[2016-08-24 16:09:13,627][TRACE][shield.authc.esnative ] [es-node-1] could not retrieve user [test1] because security index does not exist
[.security] IndexNotFoundException[no such index]
at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:151)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
[2016-08-24 16:09:13,629][DEBUG][shield.authc.esusers ] [es-node-1] user not found in cache, proceeding with normal authentication
[2016-08-24 16:09:16,181][TRACE][shield.authc.esnative ] [es-node-1] cannot poll for user changes since security index [.security] does not exist
[2016-08-24 16:27:37,761][DEBUG][shield.authc.esnative ] [es-node-1] user not found in cache, proceeding with normal authentication
[2016-08-24 16:27:37,761][TRACE][shield.authc.esnative ] [es-node-1] could not retrieve user [admin] because security index does not exist
[.security] IndexNotFoundException[no such index]
at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:151)
at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:95)
at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteSingleIndex(IndexNameExpressionResolver.java:208)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
[2016-08-24 16:27:37,762][DEBUG][shield.authc.esusers ] [es-node-1] user not found in cache, proceeding with normal authentication
[2016-08-24 16:27:37,892][DEBUG][shield.authc.esusers ] [es-node-1] authenticated user [admin], with roles []
[2016-08-24 16:27:46,196][TRACE][shield.authc.esnative ] [es-node-1] cannot poll for user changes since security index [.security] does not exist
[2016-08-24 20:05:04,561][INFO ][node ] [es-node-1] stopping ...
[2016-08-24 20:05:04,567][INFO ][watcher ] [es-node-1] stopping watch service...
[2016-08-24 20:05:05,009][INFO ][watcher.watch ] [es-node-1] stopped watch store
[2016-08-24 20:05:05,009][INFO ][watcher ] [es-node-1] watch service has stopped
[2016-08-24 20:05:05,552][INFO ][node ] [es-node-1] stopped
[2016-08-24 20:05:05,553][INFO ][node ] [es-node-1] closing ...
[2016-08-24 20:05:05,558][INFO ][node ] [es-node-1] closed
Do you have leading spaces in your configuration on the shield line? There should be messages about active directory but they aren't there
Yes, PFA better snapshot of it:
I couldn't see any error in the log file and elasticsearch is also running fine.
Hello Jay,
I tried with LDAP, and it actually tried to connect AD server but failed due to secure connection error. PFA snapshot.
I can connect to AD server from ES instance but SSL is not enable, so i believe ES is trying to make a secure connection and that's why it's failing.
Is there a way to make not secure connection with AD?
Regards...
Hi,
Please remove the leading spaces before Shield. Also, rather than screenshots please copy the text in using a code block as it allows us to more easily provide suggestions. It should look like this (no leading space in front of shield).
shield:
authc:
realms:Hi Jay,
It's yml file, if i removed space i will get error, PFB below:
Please let me know how can i make no-secure connection between ES and Active directory? it looks like by default it is trying to make secure connection, which is failing. (my observation)
Removed 2 space:
Likely root cause: expected '', but found BlockMappingStart
in 'reader', line 33, column 1:
shield:
^
at com.fasterxml.jackson.dataformat.yaml.snakeyaml.parser.ParserImpl$ParseDocumentStart.produce(ParserImpl.java:225)
at com.fasterxml.jackson.dataformat.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:158)
at com.fasterxml.jackson.dataformat.yaml.snakeyaml.parser.ParserImpl.getEvent(ParserImpl.java:168)
at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:342)
at org.elasticsearch.common.xcontent.json.JsonXContentParser.nextToken(JsonXContentParser.java:53)
at org.elasticsearch.common.settings.loader.XContentSettingsLoader.load(XContentSettingsLoader.java:72)
at org.elasticsearch.common.settings.loader.XContentSettingsLoader.load(XContentSettingsLoader.java:45)
at org.elasticsearch.common.settings.loader.YamlSettingsLoader.load(YamlSettingsLoader.java:46)
at org.elasticsearch.common.settings.Settings$Builder.loadFromStream(Settings.java:1071)
at org.elasticsearch.common.settings.Settings$Builder.loadFromPath(Settings.java:1058)
at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:88)
at org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Refer to the log for complete error details.
[FAILED]
Removed 1 space:
Likely root cause: expected '', but found BlockMappingStart
in 'reader', line 33, column 2:
shield:
^
at com.fasterxml.jackson.dataformat.yaml.snakeyaml.parser.ParserImpl$ParseDocumentStart.produce(ParserImpl.java:225)
at com.fasterxml.jackson.dataformat.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:158)
at com.fasterxml.jackson.dataformat.yaml.snakeyaml.parser.ParserImpl.getEvent(ParserImpl.java:168)
at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:342)
at org.elasticsearch.common.xcontent.json.JsonXContentParser.nextToken(JsonXContentParser.java:53)
at org.elasticsearch.common.settings.loader.XContentSettingsLoader.load(XContentSettingsLoader.java:72)
at org.elasticsearch.common.settings.loader.XContentSettingsLoader.load(XContentSettingsLoader.java:45)
at org.elasticsearch.common.settings.loader.YamlSettingsLoader.load(YamlSettingsLoader.java:46)
at org.elasticsearch.common.settings.Settings$Builder.loadFromStream(Settings.java:1071)
at org.elasticsearch.common.settings.Settings$Builder.loadFromPath(Settings.java:1058)
at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:88)
at org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:202)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:241)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Refer to the log for complete error details.
[FAILED]
Thanks & Regards
This is the current error i am getting, using LDAP:
*There is no SSL configured on AD server.
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Unable to verify an attempt to to establish a secure connection to '--------:389' because an unexpected error was encountered during validation processing: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at com.unboundid.ldap.sdk.LDAPConnectionInternals.(LDAPConnectionInternals.java:158)
at com.unboundid.ldap.sdk.LDAPConnection.connect(LDAPConnection.java:859)
... 68 more
[2016-08-26 09:36:12,537][DEBUG][shield.authc.ldap ] [es-node-1] authentication failed for user [test1]
java.io.IOException: failed to create a connection pool for realm [ldap1] as no LDAP servers are available
at org.elasticsearch.shield.authc.ldap.LdapUserSearchSessionFactory.connectionPool(LdapUserSearchSessionFactory.java:82)
at org.elasticsearch.shield.authc.ldap.LdapUserSearchSessionFactory.findUserDN(LdapUserSearchSessionFactory.java:166)
The yml errors may be because of the watcher: line above that start with a space. Unless something should be nested, there should be no leading spaces.
Also, it is trying to use ssl as you specified the URL with ldaps; use ldap for a plaintext connection.
Thanks Jay, i tried ldap and got below error:
[2016-08-26 09:47:12,018][TRACE][shield.authc.esnative ] [es-node-1] cannot poll for user changes since security index [.security] does not exist
[2016-08-26 09:47:23,816][DEBUG][shield.authc.ldap ] [es-node-1] user not found in cache, proceeding with normal authentication
[2016-08-26 09:47:23,821][DEBUG][shield.authc.ldap ] [es-node-1] unable to create connection pool for realm [ldap1]
LDAPException(resultCode=49 (invalid credentials), errorMessage='80090308: LdapErr: DSID-0C0903D0, comment: AcceptSecurityContext error, data 52e, v2580', diagnosticMessage='80090308: LdapErr: DSID-0C0903D0, comment: AcceptSecurityContext error, data 52e, v2580')
at com.unboundid.ldap.sdk.LDAPConnection.bind(LDAPConnection.java:2178)
at com.unboundid.ldap.sdk.LDAPConnectionPool.createConnection(LDAPConnectionPool.java:1138)
at com.unboundid.ldap.sdk.LDAPConnectionPool.(LDAPConnectionPool.java:1026)
at com.unboundid.ldap.sdk.LDAPConnectionPool.(LDAPConnectionPool.java:913)
at com.unboundid.ldap.sdk.LDAPConnectionPool.(LDAPConnectionPool.java:854)
at com.unboundid.ldap.sdk.LDAPConnectionPool.(LDAPConnectionPool.java:801)
at com.unboundid.ldap.sdk.LDAPConnectionPool.(LDAPConnectionPool.java:760)
at org.elasticsearch.shield.authc.ldap.LdapUserSearchSessionFactory.createConnectionPool(LdapUserSearchSessionFactory.java:101)
at org.elasticsearch.shield.authc.ldap.LdapUserSearchSessionFactory.connectionPool(LdapUserSearchSessionFactory.java:79)
at java.lang.Thread.run(Thread.java:745)
[2016-08-26 09:47:23,821][DEBUG][shield.authc.ldap ] [es-node-1] authentication failed for user [test1]
java.io.IOException: failed to create a connection pool for realm [ldap1] as no LDAP servers are available
at org.elasticsearch.shield.authc.ldap.LdapUserSearchSessionFactory.connectionPool(LdapUserSearchSessionFactory.java:82)
at org.elasticsearch.shield.authc.ldap.LdapUserSearchSessionFactory.findUserDN(LdapUserSearchSessionFactory.java:166)
at org.elasticsearch.shield.authc.ldap.LdapUserSearchSessionFactory.session(LdapUserSearchSessionFactory.java:140)
at org.elasticsearch.shield.authc.ldap.support.AbstractLdapRealm.doAuthenticate(AbstractLdapRealm.java:50)
at org.elasticsearch.shield.authc.support.CachingUsernamePasswordRealm.authenticate(CachingUsernamePasswordRealm.java:92)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
[2016-08-26 09:47:42,018][TRACE][shield.authc.esnative ] [es-node-1] cannot poll for user changes since security index [.security] does not exist
[2016-08-26 09:48:12,019][TRACE][shield.authc.esnative ] [es-node-1] cannot poll for user changes since security index [.security] does not exist
[2016-08-26 09:48:42,019][TRACE][shield.authc.esnative ] [es-node-1] cannot poll for user changes since security index [.security] does not exist
I think active directory should be easier to setup. I noticed something else in your config formatting, active_directory needs to be indented beyond realms like:
shield:
authc:
realms:
myad:
type: active_directory
url: ...Bingo, that worked. Thanks Jai.
Now my user is getting authenticated and shield also pulled all AD realm but it's not mapped to any roles, PFB logs roles =[[]].
Logs:
[2016-08-26 10:29:37,351][DEBUG][shield.authc.support ] [es-node-1] the roles [[]], are mapped from the user [active_directory] for realm [CN=test1,OU=users,OU=ops,OU=ccs,DC=,DC=,DC=,DC=com/active_directory]
[2016-08-26 10:29:37,355][DEBUG][shield.authc.activedirectory] [es-node-1] authenticated user [test1], with roles [[]]
For role mapping i configured role_mapping.yml file. I am assuming below should assign ES admin role to all AD users associated with group gp-ops-es-admin, in our case user "test1"
role_mapping.yml:
admin:
role_mapping.yml file is in the CONF_DIR. and not sure where to configure: shield.authc.active_directory.files.role_mapping
It's resolved, Thank very much Jai, for your immediate assistance.
configured:
unmapped_groups_as_roles: true
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.