I am trying to integrate our product which has an embedded node client with Shield. Yes, it is a node client and I read that we should be using a transport client, but we are looking to move to a transport client in future release. Versions are : ES v1.4.3 + Shield v1.1.1.
As soon as my application starts up, it throws :
Caused by: org.elasticsearch.shield.authz.AuthorizationException: action [indices:admin/exists] is unauthorized for user [__es_system_user]
...but it doesn't have any effect. I understand that this will allow any anonymous user have full admin control over the cluster, but I will lock anonymous access down to a lower range of privileges once I get it to work.
To get past this, I created a user in Shield and assigned that user to the admin role and then added a line to my elasticsearch.yml file :
shield.user: username:password
... and this worked. No issues, I can index data, search data, etc.
Have I misunderstood this "anonymous" access, how to configure for it and what it provides ?
You're understanding of anonymous access is correct for transport clients and http clients, but the node client behaves differently.
The node client is different than the other clients because it is also a node in the cluster and every request that leaves the node client without a user attached will have the system user attached to the request with a limited set of permissions to perform the operations needed as part of being in the cluster.
When it comes to authenticating at the next node, we look for a user attached to the request first, next we look for authentication credentials and try to authenticate, and if the request is still not authenticated we try to apply the anonymous user if it is enabled.
Now you may be wondering why shield.user works; the node client will actually authenticate the requests you make inside your application and attach that user to the request. When the request leaves the node client, the system user is not attached since there is already a user associated with that request.
Thank you Jay for explaining, and it does provide some more insight into the node to node communication which I didn't know was happening.
However, since you are saying that in "anonymous mode", all requests leave the client node with the __es_system_user attached. How to I tell my master/data node in the cluster to accept this user as a valid user ?
When I configure my client node and master/data node the same re anonymous access - I keep getting the AuthorizationException: when the client node tries to tell the master to do anything.
Unfortunately, currently we don't allow changing the permissions for the system user via a setting, and there is no way to disable attaching the system user in "anonymous mode" for node clients. Anonymous access not working with node clients is a limitation that we will work on adding to our documentation.
By do anything, you are referring to operations your application executes?
Thank you again for the confirmation that anonymous access is not supported in Shield for node clients, this is the key bit of information I needed. I will need to put the shield.user in the node client elasticsearch.yml files and lock those files down somehow because another issue is the fact that they are in plain text which breaks all security rules in enterprise software. It would be better if the password could be encrypted, that would mean there are no plain passwords in any configuration files.
Lastly, to answer your question, yes - I am referring to operations my application executes within ES.
Thanks for the followup. We are aware of the passwords being in plaintext and how that can pose a issue. We are looking at a few ways to get around this type of issue. All the feedback that we can get on these types of concerns/issues is important to our future planning.
We are also working on adding the ability to prompt a console on startup for setting values in elasticsearch. This will help on the node side, but doesn't fix the issue for clients.
Thank you for the follow up. Passwords in plain text are a big problem for enterprise software deployments and will fail a security audit with little room for dispensation.
The ability to encrypt any passwords in the yml file with your own key would be ideal. However, the key needs to reside in a keystore and you need a password for it... so the issue moves elsewhere.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.