LDAP integrated ELK with Shield

Elastic Stack Elasticsearch
1

We are configuring ELK Shield plugin. The ELK server is integrated with LDAP server which is working fine. For Kibana, we have used Apache reverse proxy.

The problem we are facing is with the configuration of Shield plugin.
We have followed the official documentation but our Kibana dashboard is not coming up and giving error 502.
One more thing , is SSL/TLS encryption mandatory in this case ?

Thanks.

2

If you are using Shield on ES, why not in KB?

3

Hi ,

Sorry, what's KB ? ohh got it, We are evaluating the Shield.

4

Hi ,

Before configuring Shield , I have gone through its complete documentation.
As a part of configuration , I have followed the steps given in - https://www.elastic.co/guide/en/shield/current/kibana.html
We have not configured SSL/TLS encryption and as per my understanding that is not mandatory.
We will enable it later on.

But after restarting kibana , I get following error -

[root@irldxvm002 kibana]# curl http://9.126.112.35:5601
curl: (52) Empty reply from server
[root@irldxvm002 kibana]# curl http://9.126.112.35:9200
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication token for REST request [/]","header":{"WWW-Authenticate":"Basic realm="shield""}}],"type":"security_exception","reason":"missing authentication token for REST request [/]","header":{"WWW-Authenticate":"Basic realm="shield""}},"status":401}

I have gone through the logs of ELK , httpd but did not get any clue.
Let me know if you need any specific details.

Regards,
Vinod

5

Hi ,

I have made some progress and now proxy error has gone. I am getting the login page with https and able to login with with my LDAP credentials but after login I am getting security exception as shown in attached screenshot-

shield_security_exception.JPG1319×687 73.6 KB

Regards,
Vinod

6

What roles are mapped to your ldap user? It appears as though the user is missing cluster monitor permissions; take a look at the my_kibana_user role on https://www.elastic.co/guide/en/shield/current/kibana.html

7

Hi Jay,

Here is the role -

my_kibana_user:
cluster:
- monitor
indices:
- names: 'filebeat-'
privileges:
- all
- names: '.kibana'
privileges:
- all

and role mapping

kibana4_server:

8

Do you have any errors in your log from startup about invalid roles? Also, if you execute the following to increase logging and authenticate as your user, some messages about role mapping should be logged:

PUT /_cluster/settings
{
    "transient" : {
        "logger.shield.authc" : "DEBUG"
    }
}
9

Hi Jay,

I am getting this log in elasticsearch-access.log

[2016-06-20 10:12:29,894] [Chrome] [transport] [access_denied] origin_type=[rest], origin_address=[127.0.0.1], principal=[vinodar3@in.ibm.com], action=[cluster:monitor/nodes/info

Following logs in logstash.log -

{:timestamp=>"2016-06-20T10:16:40.073000+0530", :message=>"[403] {"error":{"root_cause":[{"type":"security_exception","reason":"action [cluster:monitor/nodes/info] is unauthorized for user [vinodar3@in.ibm.com]"}],"type":"security_exception","reason":"action [cluster:monitor/nodes/info] is unauthorized for user [vinodar3@in.ibm.com]"},"status":403}", :class=>"Elasticsearch::Transport::Transport::Errors::Forbidden", :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/base.rb:146:in `__raise_transport_error'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/base.rb:256:in `perform_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/http/manticore.rb:54:in `perform_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/sniffer.rb:32:in `hosts'", "org/jruby/ext/timeout/Timeout.java:147:in `timeout'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/sniffer.rb:31:in `hosts'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.15/lib/elasticsearch/transport/transport/base.rb:76:in `reload_connections!'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.5-java/lib/logstash/outputs/elasticsearch/http_client.rb:72:in `sniff!'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.5-java/lib/logstash/outputs/elasticsearch/http_client.rb:60:in `start_sniffing!'", "org/jruby/ext/thread/Mutex.java:149:in `synchronize'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.5-java/lib/logstash/outputs/elasticsearch/http_client.rb:60:in `start_sniffing!'", "org/jruby/RubyKernel.java:1479:in `loop'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.5.5-java/lib/logstash/outputs/elasticsearch/http_client.rb:59:in `start_sniffing!'"], :level=>:error}

Regards,
Vinod

10

You'll need to look in the log file of the elasticsearch node that you are connecting to. The access log does not contain the role mapping debug output

11

Thanks for the reply Jay, but I have already configuring "shield.authc: TRACE" in /etc/elasticsearch/logging.yml but it looks like somehow it is not working.

12

Did you configure it under the logger section? Did you restart elasticsearch after making the change?

13

Thanks for pointing out my stupid mistake- I can see debug logs in elasticsearch.log
Though I have done the mapping , I am getting this -

[2016-06-20 17:15:36,590][DEBUG][shield.authc.support ] [Oneg the Prober] the roles [], are mapped from these [ldap] groups [] for realm [ldap/ldap1]
[2016-06-20 17:15:36,591][DEBUG][shield.authc.support ] [Oneg the Prober] the roles [], are mapped from the user [ldap] for realm [uid=xxxx,c=in,ou=bluepages,o=ibm.com/ldap]
[2016-06-20 17:15:36,595][DEBUG][shield.authc.ldap ] [Oneg the Prober] authenticated user [vinodar3@in.ibm.com], with roles []

14

You need to use that in your role mapping rather than the mail= entries. The role mapping is done off of group DNs or the user DN.

15

Thanks , now I am able to login to kibana. But I see following in the elasticsearch logs -

[2016-06-20 18:28:44,428][DEBUG][shield.authc.ldap ] [Man-Brute] authentication failed for user [AVJBYJ744]
ElasticsearchSecurityException[failed to find user [AVJBYJ744] with search base [c=in, ou=bluepages, o=ibm.com] scope [sub_tree]]
at org.elasticsearch.shield.support.Exceptions.authenticationError(Exceptions.java:39)

And now role mapping is working fine -

[2016-06-20 18:28:46,319][DEBUG][shield.authc.ldap ] [Man-Brute] authenticated user [vinodar3@in.ibm.com], with roles [[admin, kibana4_server, power_user, kibana4, my_kibana_user]]

16

Is this user AVJBYJ744 in LDAP? What does your realm configuration look like?

17

Yes that user is LDAP user and here is the realm configuration in elasticsearch.yml

shield:
authc:
realms:
ldap1:
type: ldap
order: 0
url: "ldaps://bluepages.ibm.com:636"
user_search:
base_dn: "c=in, ou=bluepages, o=ibm.com"
attribute: mail
group_search:
base_dn: "c=in, ou=bluepages, o=ibm.com"
files:
role_mapping: "/etc/elasticsearch/shield/role_mapping.yml"
unmapped_groups_as_roles: false
user_search.pool.health_check.enabled: false

shield.audit.enabled: true

18

You are using the mail attribute but the username is not a email address. I think that is why the user cannot be found.

19

But if I use emai in role mapping file, it does not work. Then I get security exception as soon as I login to kibana.
It is strange that with this configuration everything is working fine but only getting security exception in elasticsearch logs. The log says both successfully authenticated and did not find the user.

I asked other user to login who got security exception but after assigning role in role mapping file he was able to login successfully.

20

This is the only realm you have enabled? Do you see access_granted for this user in the audit logs?

Currently, we only support the distinguished name of a ldap group or user for role mapping. mail is not part of the DN so it cannot be used for role mapping, but it can be used to find the user in ldap.

Lets say we have a user with the following:

DN: "uid=JM77456,ou=bluepages,o=ibm.com"
mail: user@in.ibm.com

With the configuration you are using, the username to use for this user would be user@in.ibm.com and the role mapping definition would be:

kibana4:

What happens is a search is executed to find a user that has the mail attribute value that matches user@in.ibm.com. Once this is found, the DN is retrieved and the user is authenticated via a bind. After the bind, groups are searched for and then roles are mapped based on the DNs of the groups and the DN of the user.