PKI Authentication can't map users

Elastic Stack Elasticsearch
1

Hello all, I am trying to setup PKI authentication on my ES cluster with X-Pack but I must have done something wrong. I have setup the roles, mappings and certificates but my filebeat (the only thing that I have tested so far), seems to only get the access of an anonymous user

/etc/elasticsearch/elasticsearch.yml

cluster:
  name: my-cluster
discovery:
  zen:
    hosts_provider: ec2
    minimum_master_nodes: 3
  ec2:
    endpoint: ec2.us-west-2.amazonaws.com
    groups: sg-509a6529
    availability_zones: us-west-2a,us-west-2b
    tag.Role: es-cluster
cloud:
  node:
    auto_attributes: true
path:
  data: /var/lib/elasticsearch
  logs: /var/log/elasticsearch
xpack.security.audit.enabled: true
network:
  host: _ec2:privateIp_
script:
  allowed_types: inline
node.name: i-0c583529e070fb2e9
xpack:
  security:
    transport.ssl:
      enabled: true
      verification_mode: certificate 
      key: /etc/elasticsearch/certs/es_cert.key
      certificate: /etc/elasticsearch/certs/es_cert.crt
      certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]
    http.ssl:
      enabled: true 
      client_authentication: optional
      verification_mode: certificate 
      key: /etc/elasticsearch/certs/es_cert.key
      certificate: /etc/elasticsearch/certs/es_cert.crt
      certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]
    authc:
      realms:
        pki1:
          order: 0
          type: pki
          certificate_authorities: [ "/etc/elasticsearch/certs/ca.crt" ]
          files.role_mapping: /etc/elasticsearch/x-pack/role_mapping.yml
      anonymous.username: anonymous_user
      anonymous.roles: monitor_jobs
2

/etc/elasticsearch/x-pack/roles.yml
# All cluster rights
# All operations on all indices
admin:
cluster:
- all
indices:
- names: '*'
privileges:
- all

# monitoring cluster privileges
# All operations on all indices
power_user:
  cluster:
    - monitor
  indices:
    - names: '*'
      privileges:
        - all

# Defines permissions for filebeat containers
filebeat_container:
  cluster:
    - monitor
  indices:
    - names: 'filebeat-*'
      privileges:
        - write
        - create_index

# Defines permissions for auditlogs middleware container
es_reader:
  cluster:
    - transport_client
  indices:
    - names: '*'
      privileges:
        - read

monitor_jobs:
  cluster:
    - monitor

etc/elasticsearch/x-pack/role_mapping.yml
filebeat_container:
- "cn=filebeat.int.myorg.com"
- "c=US, st=Tennessee, l=Memphis, o=My Organization, ou=IT Dev, cn=filebeat.int.myorg.com"
- "cn=filebeats,ou=int,ou=int,ou=myorg,o=com"
auditlog_mw_container:
- "cn=middleware.int.myorg.com"
- "c=US, st=Tennessee, l=Memphis, o=My Organization, ou=IT Dev, cn=middleware.int.myorg.com"
- "cn=middleware,ou=int,ou=int,ou=myorg,o=com"
admin:
- "cn=es.int.myorg.com"
- "c=US, st=Tennessee, l=Memphis, o=My Organization, ou=IT Dev, cn=es.int.myorg.com"
- "cn=es,ou=int,ou=int,ou=myorg,o=com"

openssl x509 -in filebeat.crt -noout -text
Certificate:
    Data:
        Version: 1 (0x0)
        Serial Number: 11937562650760635722 (0xa5aabdfba28ee54a)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, ST=Tennessee, L=Memphis, O=My Organization, OU=IT Dev, CN=filebeat.int.myorg.com
        Validity
            Not Before: Feb 16 17:07:56 2018 GMT
            Not After : Feb 14 17:07:56 2028 GMT
        Subject: C=US, ST=Tennessee, L=Memphis, O=My Organization, OU=IT Dev, CN=ffilebeat.int.myorg.com/emailAddress=filebeat@int.myorg.com
3

Everytime that filebeats tries to authenticate, I get the following
/var/log/elasticsearc/my-cluster.log
[2018-02-18T19:31:19,863][INFO ][o.e.x.s.a.s.m.NativeRoleMappingStore] [i-0c583529e070fb2e9] The security index is not yet available - no role mappings can be loaded
[2018-02-18T19:31:20,277][INFO ][o.e.x.s.a.s.m.NativeRoleMappingStore] [i-0c583529e070fb2e9] The security index is not yet available - no role mappings can be loaded
[2018-02-18T19:32:20,603][INFO ][o.e.x.s.a.s.m.NativeRoleMappingStore] [i-0c583529e070fb2e9] The security index is not yet available - no role mappings can be loaded
[2018-02-18T19:32:20,846][INFO ][o.e.x.s.a.s.m.NativeRoleMappingStore] [i-0c583529e070fb2e9] The security index is not yet available - no role mappings can be loaded
[2018-02-18T19:32:21,535][INFO ][o.e.x.s.a.s.m.NativeRoleMappingStore] [i-0c583529e070fb2e9] The security index is not yet available - no role mappings can be loaded
[2018-02-18T19:33:21,725][INFO ][o.e.x.s.a.s.m.NativeRoleMappingStore] [i-0c583529e070fb2e9] The security index is not yet available - no role mappings can be loaded
[2018-02-18T19:33:21,952][INFO ][o.e.x.s.a.s.m.NativeRoleMappingStore] [i-0c583529e070fb2e9] The security index is not yet available - no role mappings can be loaded
[2018-02-18T19:33:22,611][INFO ][o.e.x.s.a.s.m.NativeRoleMappingStore] [i-0c583529e070fb2e9] The security index is not yet available - no role mappings can be loaded
[2018-02-18T19:34:22,802][INFO ][o.e.x.s.a.s.m.NativeRoleMappingStore] [i-0c583529e070fb2e9] The security index is not yet available - no role mappings can be loaded
[2018-02-18T19:34:23,027][INFO ][o.e.x.s.a.s.m.NativeRoleMappingStore] [i-0c583529e070fb2e9] The security index is not yet available - no role mappings can be loaded
[2018-02-18T19:34:23,975][INFO ][o.e.x.s.a.s.m.NativeRoleMappingStore] [i-0c583529e070fb2e9] The security index is not yet available - no role mappings can be loaded

/var/log/elasticsearch/my-cluster_access.log
> > [2018-02-18T19:31:19,678] [transport] [access_granted] origin_type=[rest], origin_address=[10.158.4.5], principal=[filebeat.int.myorg.com], roles=[monitor_jobs], action=[cluster:monitor/main], request=[MainRequest]
> > [2018-02-18T19:31:19,863] [transport] [access_denied] origin_type=[rest], origin_address=[10.158.4.5], principal=[filebeat.int.myorg.com], roles=[monitor_jobs], action=[indices:admin/template/get], request=[GetIndexTemplatesRequest]
> > [2018-02-18T19:31:20,279] [transport] [access_denied] origin_type=[rest], origin_address=[10.158.4.5], principal=[filebeat.int.myorg.com], roles=[monitor_jobs], action=[indices:admin/template/put], indices=[filebeat-6.2.1-], request=[PutIndexTemplateRequest]
> > [2018-02-18T19:32:20,603] [transport] [access_granted] origin_type=[rest], origin_address=[10.158.4.5], principal=[filebeat.int.myorg.com], roles=[monitor_jobs], action=[cluster:monitor/main], request=[MainRequest]
> > [2018-02-18T19:32:20,846] [transport] [access_denied] origin_type=[rest], origin_address=[10.158.4.5], principal=[filebeat.int.myorg.com], roles=[monitor_jobs], action=[indices:admin/template/get], request=[GetIndexTemplatesRequest]
> > [2018-02-18T19:32:21,537] [transport] [access_denied] origin_type=[rest], origin_address=[10.158.4.5], principal=[filebeat.int.myorg.com], roles=[monitor_jobs], action=[indices:admin/template/put], indices=[filebeat-6.2.1-], request=[PutIndexTemplateRequest]
> > [2018-02-18T19:33:21,726] [transport] [access_granted] origin_type=[rest], origin_address=[10.158.4.5], principal=[filebeat.int.myorg.com], roles=[monitor_jobs], action=[cluster:monitor/main], request=[MainRequest]
> > [2018-02-18T19:33:21,952] [transport] [access_denied] origin_type=[rest], origin_address=[10.158.4.5], principal=[filebeat.int.myorg.com], roles=[monitor_jobs], action=[indices:admin/template/get], request=[GetIndexTemplatesRequest]
> > [2018-02-18T19:33:22,613] [transport] [access_denied] origin_type=[rest], origin_address=[10.158.4.5], principal=[filebeat.int.myorg.com], roles=[monitor_jobs], action=[indices:admin/template/put], indices=[filebeat-6.2.1-], request=[PutIndexTemplateRequest]
> > [2018-02-18T19:34:22,802] [transport] [access_granted] origin_type=[rest], origin_address=[10.158.4.5], principal=[filebeat.int.myorg.com], roles=[monitor_jobs], action=[cluster:monitor/main], request=[MainRequest]
> > [2018-02-18T19:34:23,028] [transport] [access_denied] origin_type=[rest], origin_address=[10.158.4.5], principal=[filebeat.int.myorg.com], roles=[monitor_jobs], action=[indices:admin/template/get], request=[GetIndexTemplatesRequest]
> > [2018-02-18T19:34:23,977] [transport] [access_denied] origin_type=[rest], origin_address=[10.158.4.5], principal=[filebeat.int.myorg.com], roles=[monitor_jobs], action=[indices:admin/template/put], indices=[filebeat-6.2.1-], request=[PutIndexTemplateRequest]

Please help.

4

It is helpful if you tell us what version if Elasticsearch you are running as the details can vary quite significantly between releases.

There's a note at the bottom of the PKI realm documentation that will be helpful for resolving this problem:

The disinguished name for a PKI user follows X.500 naming conventions which place the most specific fields (like cn or uid) at the beginning of the name, and the most general fields (like o or dc) at the end of the name. Some tools, such as openssl, may print out the subject name in a different format.

One way that you can determine the correct DN for a certificate is to use the authenticate API (use the relevant PKI certificate as the means of authentication) and inspect the metadata field in the result. The user’s distinguished name will be populated under the pki_dn key. You can also use the authenticate API to validate your role mapping.

That implies you have no security index, which also means you have not set the password for the builtin users, and you should do that as a matter of priority. The steps for that are dependent on the version of Elasticsearch you are running.

5

TimV,
It looks like the default password not being set was my entire problem. Thank you very much.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.