Shield Elasticsearch & logstash cluster health monitor security exception

Elastic Stack Elasticsearch
1

Hi there,

I am trying to test the shield configuration with in elasticsearch facing an issue.

License and Shield have been installed on elasticsearch host.
LDAP user created as "logstash"
LDAP realm has been updated in "elasticsearch.yml"

When I tried to make a curl request on cluster health the following exception is thrown. I am wondering whether there is a problem in my role_mapping.yml file or some thing else is causing the issue.

'http://localhost:9200/_cluster/health?pretty'
{
"error" : {
"root_cause" : [ {
"type" : "security_exception",
"reason" : "action [cluster:monitor/health] is unauthorized for user [logstash]"
} ],
"type" : "security_exception",
"reason" : "action [cluster:monitor/health] is unauthorized for user [logstash]"
},
"status" : 403

In logstash.log

:message=>"[403] {"error":{"root_cause":[{"type":"security_exception","reason":"action [indices:data/write/bulk] is unauthorized for user [logstash]"}]

Please could you let me know what is the issue here.

2

I think your role mapping could be the issue. If you set the logging to debug for shield.authc you should see a log line that prints out the DNs of the groups retrieved from ldap and what roles were mapped. https://www.elastic.co/guide/en/shield/current/troubleshooting.html#_ldap

3

Thanks for the reply. Please could you go through the below configure and find if any thing wrong.

elasticsearch version : 2.2.0
shield : 2.2.0
logstash version : 2.2.2

As per the response, I have updated the logging to debug but after the server restart none were logged related to DN.

It looks to me authentication to LDAP is happing, in case if i pass in my curl request for health check as wrong password. it throws the below exception otherwise no messages in the log.

shield.authc.ldap ] [node-01] authentication failed for user [logstash]: failed LDAP authentication for

The only exception in my ES.log are

[INFO ][rest.suppressed ] /_bulk Params: {}
ElasticsearchSecurityException[action [indices:data/write/bulk] is unauthorized for user [logstash]]

Here is my setting for logger.yml,
logger:
shield.audit.logfile: DEBUG, access_log
shield.authc: debug

elasticsearch.yml for ldap settings

shield:
authc:
realms:
ldap1:
type: ldap
order: 0
url: "ldap://xxxx:389"
bind_dn: "uid=logstash,ou=users,dc=example,dc=com"
bind_password: "xxxxx"
user_search:
base_dn: "dc=example,dc=com"
attribute: cn
group_search:
base_dn: "dc=example,dc=com"
files:
role_mapping: "/usr/share/elasticsearch/plugins/shield/config/role_mapping.yml"
unmapped_groups_as_roles: false

role_mapping.yml, the following has been set

logstash:

  • "cn=logstash_users_group,ou=groups,dc=example,dc=com"
  • "cn=logstash,dc=example,dc=com"
4

The logger settings need to go in the elasticsearch logging file; it looks like you have them in the shield logging.yml file. It may be easier to do this:

curl -XPUT -u admin 'localhost:9200/_cluster/settings' -d '{
    "transient" : {
        "shield.authc" : "DEBUG"
    }
}'

See https://www.elastic.co/guide/en/elasticsearch/guide/current/logging.html

5

I cannot able to run the above curl request due to my esusers realm user does not have privileges. BTW just switched over esusers to run the above command. The user "esusers_user1" is part of admin, power_user, user, logstash roles.

Also is there any way can we specifiy manually this entry in the yml file rather via curl request? If so in which yml does that goes i.e elasticsearch.yml?

{"error":{"root_cause":[{"type":"security_exception","reason":"action [cluster:admin/settings/update] is unauthorized for user [esusers_user1]"}],"type":"security_exception","reason":"action [cluster:admin/settings/update] is unauthorized for user [esusers_user1]"},"status":403}

===================

I did bit more investigation, It looks to me that the RPM installation and Puppet module slightly using different approach. When I start the elasticsearch instance manually from "/usr/share/elasticsearch/bin/" by setting the export ES_JAVA_OPTS="-Des.path.conf=/etc/elasticsearch" then I can access the cluster health commands.

I will have to find the way of using these variable in the current puppet module of 0.10.3. In the mean while if you spot any thing related. Please update me.