PKI authentication in logstash Elastic Search Output not working

Elastic Stack Elasticsearch
1

Hi all, I am trying to achieve PKI auth in Logstash elastic search output plugin using shield trail license .Below are my configs

Logstash :

output {
elasticsearch {
cluster => "abc"
host => ["host1","host2:port"]
protocol : "http"
ssl => true
keystore => "/etc/pki/logstashStr.jks"
keystore_password => "*****"
cacert => "etc/pki/servercert.crt"
index => "logstash-mm-dd-yyyy"

Still the exception is coming in logstash logs as :
AuthenticationException[missing Authentication token for REST service status 401]

My ELK stack is : Kibana -4.0.3
ES - 1.7.1
logstash - 1.5.3
LSF - 0.4.0

Please help in enabling PKI . I have done required configs in Shield for PKI also .

2

What does your Shield realm configuration look like? Is there anything in the server-side logs about the authentication attempts?

index => "logstash-mm-dd-yyyy"

I think you mean this:

index => "logstash-%{+MM-dd-YYYY}"

3

Shield realm is very basic :

shield.authc.realms :
esusers:
type: esusers
order:0
pki1:
type: pki
order: 1

4

in Shield access logs i found out error where it mentioned Anonymous Access denied.

5

I get success only if i am providing user and password in logstash elasticsearch plugin but PKI auth is not working .

6

Hi,

Does certificate on logstash specify the username in the common name (CN)? And what you specified in the CN map to something in the role_mapping.yml?

7

CN of logstash certificate contains the logstash server name and in role_mapping.xml i have mapped power_user role to CN Map i.e CN=servername ,OU=CompanyUnit,O=Company.

Can you please elaborate if i need to create a user in Elasticsearch in esusers with that server name ?

8

have you enabled SSL on your cluster? Can you share all of the shield settings?

9

You should map the user to the logstash role. I don't think the power_user has sufficient privileges. See the following:

https://www.elastic.co/guide/en/shield/current/logstash.html#ls-user

Also I'd check the log file on the Elasticsearch cluster for any error messages related to SSL.

10

These are my shield configs :

shield.ssl.keystore.path: /etc/pki/elasticWild.jks
shield.ssl.keystore.password: *****
shield.http.ssl: true
shield.transport.ssl: true

shield.audit.enabled: true
######################################Cipher Config########################
shield.ssl.ciphers: TLS_RSA_WITH_AES_128_CBC_SHA

shield.authc.realms:
esusers:
type: esusers
order: 0

pki1:
type: pki
order: 1

11

As per the error in logstash logs which says Authentication token missing in the request . it seems logstash is not sending the cert details to the elastic search and elastic search is not getting any PKI cert creds . Is it really an issue with shield config ?

12

It looks like you are using the logstash HTTP output. In order for PKI authentication to work with http you will need to set:

shield.http.ssl.client.auth: optional

This setting will ask the clients to present a certificate for authentication, but it does not require the certificate so basic authentication will still work.

13

Hi I made the suggested changes in shield config , now the user is being authenticated but now i am getting Autharization Exception . Even though i have already defined the mapping in role_mapping.yml

equest=[BulkRequest]
[2015-10-26 15:27:06,435][DEBUG][shield.ssl ] no truststore defined. using keystore [/etc/pki/elasticWild.jks] as truststore
[2015-10-26 15:27:07,307][DEBUG][shield.authc.support ] [Sinister] the roles [[]], are mapped from these [pki] groups [[]] for realm [pki/pki1]
[2015-10-26 15:27:07,308][DEBUG][shield.audit.logfile ] [Sinister] [transport] [access_denied] origin_type=[rest], origin_address=[], principal=[***], action=[indices:data/write/bulk], request=[BulkRequest]
[2015-10-26 15:27:08,281][DEBUG][shield.ssl ] no truststore defined. using keystore [/etc/pki/elasticWild.jks] as truststore
[2015-10-26 15:27:09,331][DEBUG][shield.authc.support ] [Sinister] the roles [[]], are mapped from these [pki] groups [[]] for realm [pki/pki1]

14

shield.http.ssl.client.auth: optional , does this config mean that we are only authenticating the Server and bypassing the Client machine auth as part of SSL authentication ? Means we are not implementing the Mutual Authentication ?

15

For more details ,below is my details in role-mapping.yml

admin:

  • "CN=server name,OU=unitname,O=unit"

This value is matching with the CN,OU and O in the certificate and also with the principal coming in the logs

2015-10-26 15:27:07,308][DEBUG][shield.audit.logfile ] [Sinister] [transport] [access_denied] origin_type=[rest], origin_address=[], principal=[***], action=[indices:data/write/bulk], request=[BulkRequest]

16

It appears as though there is some issue with the role mapping. When there is a role mapping that matches the user DN, you would see the following line:

the roles [admin], are mapped from the user [CN=server name,OU=unitname,O=unit] for realm [pki/pki1]

Is the formatting correct in the yml file? Only spaces and no tabs? Is that the full Subject DN in your certificate?

This means that mutual authentication is not required but is requested; if the client does not provide a certificate then they will have to authenticate using another method such as username/password. You can change optional to required to require all HTTP clients to perform mutual authentication.

17

I checked the role mapping file and i have only spaces and no tabs and i have added below entry :

admin:
-"CN=servername,OU=unitname,O=unit"

and it matches with the certificate Subject DN .

18

1 more thing , can we add 2 Subject DNs in 1 role like below :

admin:

  • "CN=server1,OU=unitname,O=unit"
  • "CN=server2,OU=unitname,O=unit"

Although currently i am using only 1 subject DN.

19

Yes, you can have multiple subject DNs like in your example. Were you able to get it working?

20

Hi Jay, I could achieve the PKI auth but Authorization still not working it is not able to get the role from the role-mapping.