# PKI authentication in logstash Elastic Search Output not working **URL:** \ **Category:** Elasticsearch\ **Created:** [October 22, 2015, 3:19am UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738 "2015-10-22T03:19:39Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)\ **Post date:** [October 22, 2015, 3:19am UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/1 "2015-10-22T03:19:39Z")
Hi all, I am trying to achieve PKI auth in Logstash elastic search output plugin using shield trail license .Below are my configs Logstash : output { elasticsearch { cluster =\> "abc" host =\> ["host1","host2:port"] protocol : "http" ssl =\> true keystore =\> "/etc/pki/logstashStr.jks" keystore\_password =\> "\*\*\*\*\*" cacert =\> "etc/pki/servercert.crt" index =\> "logstash-mm-dd-yyyy" Still the exception is coming in logstash logs as : AuthenticationException[missing Authentication token for REST service status 401] My ELK stack is : Kibana -4.0.3 ES - 1.7.1 logstash - 1.5.3 LSF - 0.4.0 Please help in enabling PKI . I have done required configs in Shield for PKI also . ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 22, 2015, 6:51am UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/2 "2015-10-22T06:51:27Z")
What does your Shield realm configuration look like? Is there anything in the server-side logs about the authentication attempts? > index =\> "logstash-mm-dd-yyyy" I think you mean this: index =\> "logstash-%{+MM-dd-YYYY}" ---
**Author:** ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)\ **Post date:** [October 22, 2015, 5:15pm UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/3 "2015-10-22T17:15:05Z")
Shield realm is very basic : shield.authc.realms : esusers: type: esusers order:0 pki1: type: pki order: 1 ---
**Author:** ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)\ **Post date:** [October 22, 2015, 5:16pm UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/4 "2015-10-22T17:16:22Z")
in Shield access logs i found out error where it mentioned Anonymous Access denied. ---
**Author:** ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)\ **Post date:** [October 22, 2015, 8:30pm UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/5 "2015-10-22T20:30:17Z")
I get success only if i am providing user and password in logstash elasticsearch plugin but PKI auth is not working . ---
**Author:** ![msimos](https://avatars.discourse-cdn.com/v4/letter/m/bb73d2/32.png) [@msimos](https://discuss.elastic.co/u/msimos)\ **Post date:** [October 22, 2015, 9:47pm UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/6 "2015-10-22T21:47:24Z")
Hi, Does certificate on logstash specify the username in the common name (CN)? And what you specified in the CN map to something in the role\_mapping.yml? ---
**Author:** ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)\ **Post date:** [October 23, 2015, 12:16am UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/7 "2015-10-23T00:16:19Z")
CN of logstash certificate contains the logstash server name and in role\_mapping.xml i have mapped power\_user role to CN Map i.e CN=servername ,OU=CompanyUnit,O=Company. Can you please elaborate if i need to create a user in Elasticsearch in esusers with that server name ? ---
**Author:** ![jaymode](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaymode/32/50103_2.png) [@jaymode](https://discuss.elastic.co/u/jaymode)\ **Post date:** [October 23, 2015, 12:06pm UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/8 "2015-10-23T12:06:36Z")
have you enabled SSL on your cluster? Can you share all of the shield settings? ---
**Author:** ![msimos](https://avatars.discourse-cdn.com/v4/letter/m/bb73d2/32.png) [@msimos](https://discuss.elastic.co/u/msimos)\ **Post date:** [October 23, 2015, 8:38pm UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/9 "2015-10-23T20:38:13Z")
You should map the user to the logstash role. I don't think the power\_user has sufficient privileges. See the following: [https://www.elastic.co/guide/en/shield/current/logstash.html#ls-user](https://www.elastic.co/guide/en/shield/current/logstash.html#ls-user) Also I'd check the log file on the Elasticsearch cluster for any error messages related to SSL. ---
**Author:** ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)\ **Post date:** [October 23, 2015, 9:20pm UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/10 "2015-10-23T21:20:05Z")
These are my shield configs : shield.ssl.keystore.path: /etc/pki/elasticWild.jks shield.ssl.keystore.password: \*\*\*\*\* shield.http.ssl: true shield.transport.ssl: true shield.audit.enabled: true ######################################Cipher Config######################## shield.ssl.ciphers: TLS\_RSA\_WITH\_AES\_128\_CBC\_SHA shield.authc.realms: esusers: type: esusers order: 0 pki1: type: pki order: 1 ---
**Author:** ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)\ **Post date:** [October 24, 2015, 4:27am UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/11 "2015-10-24T04:27:57Z")
As per the error in logstash logs which says Authentication token missing in the request . it seems logstash is not sending the cert details to the elastic search and elastic search is not getting any PKI cert creds . Is it really an issue with shield config ? ---
**Author:** ![jaymode](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaymode/32/50103_2.png) [@jaymode](https://discuss.elastic.co/u/jaymode)\ **Post date:** [October 26, 2015, 12:18pm UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/12 "2015-10-26T12:18:44Z")
It looks like you are using the logstash HTTP output. In order for PKI authentication to work with http you will need to set: `shield.http.ssl.client.auth: optional` This setting will ask the clients to present a certificate for authentication, but it does not require the certificate so basic authentication will still work. ---
**Author:** ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)\ **Post date:** [October 26, 2015, 10:30pm UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/13 "2015-10-26T22:30:16Z")
Hi I made the suggested changes in shield config , now the user is being authenticated but now i am getting Autharization Exception . Even though i have already defined the mapping in role\_mapping.yml equest=[BulkRequest] [2015-10-26 15:27:06,435][DEBUG][shield.ssl] no truststore defined. using keystore [/etc/pki/elasticWild.jks] as truststore [2015-10-26 15:27:07,307][DEBUG][shield.authc.support] [Sinister] the roles [[]], are mapped from these [pki] groups [[]] for realm [pki/pki1] [2015-10-26 15:27:07,308][DEBUG][shield.audit.logfile] [Sinister] [transport] [access\_denied] origin\_type=[rest], origin\_address=[**_], principal=[_**\*\*\*], action=[indices:data/write/bulk], request=[BulkRequest] [2015-10-26 15:27:08,281][DEBUG][shield.ssl] no truststore defined. using keystore [/etc/pki/elasticWild.jks] as truststore [2015-10-26 15:27:09,331][DEBUG][shield.authc.support] [Sinister] the roles [[]], are mapped from these [pki] groups [[]] for realm [pki/pki1] ---
**Author:** ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)\ **Post date:** [October 26, 2015, 10:32pm UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/14 "2015-10-26T22:32:47Z")
shield.http.ssl.client.auth: optional , does this config mean that we are only authenticating the Server and bypassing the Client machine auth as part of SSL authentication ? Means we are not implementing the Mutual Authentication ? ---
**Author:** ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)\ **Post date:** [October 26, 2015, 11:37pm UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/15 "2015-10-26T23:37:06Z")
For more details ,below is my details in role-mapping.yml admin: - "CN=server name,OU=unitname,O=unit" This value is matching with the CN,OU and O in the certificate and also with the principal coming in the logs 2015-10-26 15:27:07,308][DEBUG][shield.audit.logfile] [Sinister] [transport] [access\_denied] origin\_type=[rest], origin\_address=[**_], principal=[_**\*\*\*], action=[indices:data/write/bulk], request=[BulkRequest] ---
**Author:** ![jaymode](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaymode/32/50103_2.png) [@jaymode](https://discuss.elastic.co/u/jaymode)\ **Post date:** [October 27, 2015, 10:49am UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/16 "2015-10-27T10:49:45Z")
It appears as though there is some issue with the role mapping. When there is a role mapping that matches the user DN, you would see the following line: ```auto the roles [admin], are mapped from the user [CN=server name,OU=unitname,O=unit] for realm [pki/pki1] ``` Is the formatting correct in the yml file? Only spaces and no tabs? Is that the full Subject DN in your certificate? > [@soodlikesjava](#): > > shield.http.ssl.client.auth: optional , does this config mean that we are only authenticating the Server and bypassing the Client machine auth as part of SSL authentication This means that mutual authentication is not required but is requested; if the client does not provide a certificate then they will have to authenticate using another method such as username/password. You can change `optional` to `required` to require all HTTP clients to perform mutual authentication. ---
**Author:** ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)\ **Post date:** [October 30, 2015, 8:26pm UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/17 "2015-10-30T20:26:27Z")
I checked the role mapping file and i have only spaces and no tabs and i have added below entry : admin: -"CN=servername,OU=unitname,O=unit" and it matches with the certificate Subject DN . ---
**Author:** ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)\ **Post date:** [October 30, 2015, 8:28pm UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/18 "2015-10-30T20:28:15Z")
1 more thing , can we add 2 Subject DNs in 1 role like below : admin: - "CN=server1,OU=unitname,O=unit" - "CN=server2,OU=unitname,O=unit" Although currently i am using only 1 subject DN. ---
**Author:** ![jaymode](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaymode/32/50103_2.png) [@jaymode](https://discuss.elastic.co/u/jaymode)\ **Post date:** [November 10, 2015, 11:44am UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/19 "2015-11-10T11:44:48Z")
Yes, you can have multiple subject DNs like in your example. Were you able to get it working? ---
**Author:** ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)\ **Post date:** [November 17, 2015, 5:53pm UTC](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738/20 "2015-11-17T17:53:04Z")
Hi Jay, I could achieve the PKI auth but Authorization still not working it is not able to get the role from the role-mapping. [Next page](https://discuss.elastic.co/t/pki-authentication-in-logstash-elastic-search-output-not-working/32738.md?page=2)