Unable to authenticate user [es_admin] for REST request [/]

Elastic Stack Elasticsearch
1

I am using elasticsearch 2.3.3 and kibana 4.5. Also configured shield.
I can authenticate localhost:9200 with es_admin user and kibana with a kibana4-server user.
Now I configured shield for active directory users. The steps that I followed are

  1. Created a Usergroup in ActiveDirectory - UGElasticsearch with group scope domain local

  2. Created a User - ESUser (Firstname:Elasticsearch ,Logon name: ESUser)

  3. Updated the UGElasticsearch with member ESUser
    4.In elasticsearch.yml (at the bottom of the page)

    shield:
    authc:
    realms:
    active_directory:
    type: active_directory
    order: 0
    domain_name: DC07.Loc
    url: ldaps://C4968397007.DC07.Loc
    unmapped_groups_as_roles: true

  4. In role_mapping.yml

        admins
           - "CN=UGElasticsearch,OU=Groups,DC=DC07,DC=Loc"

  5. Restarted elasticsearch

  6. In browser opened http://localhost:9200, a login screen came with username es_admin

I could not login as es_admin or ESUser.
Am I done any mistake?

2

Maybe there is some hint as to why authentication is failing in the elasticsearch log. Do you see any exceptions?

3

There is no exception in the log file.

elastisearch.log file logs

 [2016-06-23 01:47:09,548][INFO ][node                     ] [Hardcase] version[2.3.3], pid[17116], build[218bdf1/2016-05-17T15:40:04Z]
  [2016-06-23 01:47:09,549][INFO ][node                     ] [Hardcase] initializing ...
 [2016-06-23 01:47:11,358][INFO ][plugins                  ] [Hardcase] modules [reindex, lang-expression, lang-groovy], plugins [license, shield, mapper-attachments], sites []
   [2016-06-23 01:47:11,452][INFO ][env                      ] [Hardcase] using [1] data paths, mounts [[(C:)]], net usable_space [4.8gb], net total_space [124.6gb], spins? [unknown], types [NTFS]
 [2016-06-23 01:47:11,452][INFO ][env                      ] [Hardcase] heap size [989.8mb], compressed ordinary object pointers [true]
 [2016-06-23 01:47:12,694][INFO ][http                     ] [Hardcase] Using [org.elasticsearch.http.netty.NettyHttpServerTransport] as http transport, overridden by [shield]
 [2016-06-23 01:47:13,278][INFO ][transport                ] [Hardcase] Using [org.elasticsearch.shield.transport.ShieldServerTransportService] as transport service, overridden by [shield]
[2016-06-23 01:47:13,279][INFO ][transport                ] [Hardcase] Using [org.elasticsearch.shield.transport.netty.ShieldNettyTransport] as transport, overridden by [shield]
   [2016-06-23 01:47:19,162][INFO ][node                     ] [Hardcase] initialized
    [2016-06-23 01:47:19,162][INFO ][node                     ] [Hardcase] starting ...
 [2016-06-23 01:47:21,483][INFO ][shield.transport         ] [Hardcase] publish_address   {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}, {[::1]:9300}
 [2016-06-23 01:47:21,490][INFO ][discovery                ] [Hardcase] elasticsearch/MLLWOYOyRUKCkt2WwG0kvQ
 [2016-06-23 01:47:25,610][INFO ][cluster.service          ] [Hardcase] new_master {Hardcase}{MLLWOYOyRUKCkt2WwG0kvQ}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
 [2016-06-23 01:47:25,780][INFO ][license.plugin.core      ] [Hardcase] license [2918740c-aaf9-401e-8f76-cef1577be584] - valid
 [2016-06-23 01:47:25,784][ERROR][license.plugin.core      ] [Hardcase]
4

es_admin is a user that exists in the file based realm? If so you need to define a realm of that type in the chain. It is available by default until you configure another realm.

5

I haven't added anything specifically for file based realm. Below is my elasticsearch.yml file. Am I need to delete something?

======================== Elasticsearch Configuration =========================

NOTE: Elasticsearch comes with reasonable defaults for most settings.

Before you set out to tweak and tune the configuration, make sure you

understand what are you trying to accomplish and the consequences.

The primary way of configuring a node is via this file. This template lists

the most important settings you may want to configure for a production cluster.

Please see the documentation for further information on configuration options:

http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html

---------------------------------- Cluster -----------------------------------

Use a descriptive name for your cluster:

cluster.name: my-application

------------------------------------ Node ------------------------------------

Use a descriptive name for the node:

node.name: node-1

Add custom attributes to the node:

node.rack: r1

----------------------------------- Paths ------------------------------------

Path to directory where to store the data (separate multiple locations by comma):

path.data: /path/to/data

Path to log files:

path.logs: /path/to/logs

----------------------------------- Memory -----------------------------------

Lock the memory on startup:

bootstrap.mlockall: true

Make sure that the ES_HEAP_SIZE environment variable is set to about half the memory

available on the system and that the owner of the process is allowed to use this limit.

Elasticsearch performs poorly when the system is swapping the memory.

---------------------------------- Network -----------------------------------

Set the bind address to a specific IP (IPv4 or IPv6):

network.host: 192.168.0.1

Set a custom port for HTTP:

http.port: 9200

For more information, see the documentation at:

http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html

--------------------------------- Discovery ----------------------------------

Pass an initial list of hosts to perform discovery when new node is started:

The default list of hosts is ["127.0.0.1", "[::1]"]

discovery.zen.ping.unicast.hosts: ["host1", "host2"]

Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):

discovery.zen.minimum_master_nodes: 3

For more information, see the documentation at:

http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html

---------------------------------- Gateway -----------------------------------

Block initial recovery after a full cluster restart until N nodes are started:

gateway.recover_after_nodes: 3

For more information, see the documentation at:

http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html

---------------------------------- Various -----------------------------------

Disable starting multiple nodes on a single system:

node.max_local_storage_nodes: 1

Require explicit names when deleting indices:

action.destructive_requires_name: true

shield:
authc:
realms:
active_directory:
type: active_directory
order: 0
domain_name: DC07.Loc
url: ldaps://C4968397007.DC07.Loc
unmapped_groups_as_roles: true

6

Can you answer this question? If the answer is "yes" then you need to configure the file realm as part of the realm chain, see File-based User Authentication | Shield [2.4] | Elastic

7

I need Active directory for user authentication. file based realm not required.

8

Ok. Lets take Kibana out of the equation, can you try to authenticate to elasticsearch as a active directory user?

curl localhost:9200 -u your_user

After that, is there anything in the logs? Also you use ldaps but do not specify a truststore or keystore, is the certificate for your ldap server signed by a root CA?

9

My elasticsearch.yml is showing an error like this. In btw, I created the certificate using OpenSSL.

-06-26 00:46:16,912][WARN ][shield.authc.activedirectory] [Solarman] authentication failed for user [es_admin]: failed to connect to any active directory servers
cause: com.unboundid.ldap.sdk.LDAPException: An error occurred while attempting to connect to server C4968397007.DC07.Loc:636:  java.io.IOException: Unable to verify an attempt to to establish a secure connection to 'C4968397007.DC07.Loc:636' because an unexpected error was encountered during validation processing:  javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
10

You need to configure a truststore so that the certificate for the LDAP server will be trusted. https://www.elastic.co/guide/en/shield/current/active-directory-realm.html#active-directory-ssl