# Elastic search + Shield instance is not authenticating logstash user
**URL:** https://discuss.elastic.co/t/elastic-search-shield-instance-is-not-authenticating-logstash-user/34891
**Category:** Elasticsearch
**Tags:** elastic-stack-security
**Created:** [November 18, 2015, 7:49am UTC](https://discuss.elastic.co/t/elastic-search-shield-instance-is-not-authenticating-logstash-user/34891 "2015-11-18T07:49:33Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@robertranjan](https://discuss.elastic.co/u/robertranjan)
#### Post date: [November 18, 2015, 7:49am UTC](https://discuss.elastic.co/t/elastic-search-shield-instance-is-not-authenticating-logstash-user/34891/1 "2015-11-18T07:49:33Z")
I'm sending events from logstash 2.0 to ES 2.0 with shield. ES is not accepting any events/logs/requests for logstash user though roles are seems correct.
## Here is my roles.yml setting:
```
logstash:
cluster: indices:admin/template/get, indices:admin/template/put
indices:
'logstash-*':
privileges: indices:data/write/bulk, indices:data/write/delete, indices:data/write/update, indices:data/read/search, indices:data/read/scroll, create_index, cluster:monitor/state
```
Still ES throws below error in elasticserch.log.
---snip---
```
[2015-11-18 06:47:20,876][INFO][rest.suppressed] /_bulk Params: {}
ElasticsearchSecurityException[action [indices:data/write/bulk] is unauthorized for user [logstash]]
at org.elasticsearch.shield.support.Exceptions.authorizationError(Exceptions.java:45)
at org.elasticsearch.shield.authz.InternalAuthorizationService.denialException(InternalAuthorizationService.java:296)
at org.elasticsearch.shield.authz.InternalAuthorizationService.denial(InternalAuthorizationService.java:270)
at org.elasticsearch.shield.authz.InternalAuthorizationService.authorize(InternalAuthorizationService.java:131)
at org.elasticsearch.shield.action.ShieldActionFilter.apply(ShieldActionFilter.java:105)
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:99)
```
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [November 18, 2015, 9:43am UTC](https://discuss.elastic.co/t/elastic-search-shield-instance-is-not-authenticating-logstash-user/34891/2 "2015-11-18T09:43:18Z")
What does your role\_mapping file look like? Check that you did not invert user and role. Role should be first.
What does your elasticsearch output plugin configuration look like?
---
### Author:  [@robertranjan](https://discuss.elastic.co/u/robertranjan)
#### Post date: [November 18, 2015, 4:45pm UTC](https://discuss.elastic.co/t/elastic-search-shield-instance-is-not-authenticating-logstash-user/34891/3 "2015-11-18T16:45:54Z")
```
[root@es1 shield]# cat /etc/elasticsearch/shield/roles.yml
# All cluster rights
# All operations on all indices
admin:
cluster: all
indices:
'*':
privileges: all
# monitoring cluster privileges
# All operations on all indices
power_user:
cluster: monitor
indices:
'*':
privileges: all
# Read-only operations on indices
user:
indices:
'*':
privileges: read
# Defines the required permissions for transport clients
transport_client:
cluster:
- cluster:monitor/nodes/liveness
#uncomment the following for sniffing
#- cluster:monitor/state
# The required role for logstash users
logstash:
cluster: indices:admin/template/get, indices:admin/template/put
indices:
'logstash-*':
privileges: indices:data/write/bulk, indices:data/write/delete, indices:data/write/update, indices:data/read/search, indices:data/read/scroll, create_index, cluster:monitor/state
# The required permissions for kibana 4 users.
kibana4:
cluster:
- cluster:monitor/nodes/info
- cluster:monitor/health
indices:
'*':
privileges: indices:admin/mappings/fields/get, indices:admin/validate/query, indices:data/read/search, indices:data/read/msearch, indices:admin/get
'.kibana':
privileges: indices:admin/exists, indices:admin/mapping/put, indices:admin/mappings/fields/get, indices:admin/refresh, indices:admin/validate/query, indices:data/read/get, indices:data/read/mget, indices:data/read/search, indices:data/write/delete, indices:data/write/index, indices:data/write/update
# The required permissions for the kibana 4 server
kibana4_server:
cluster:
- cluster:monitor/nodes/info
- cluster:monitor/health
indices:
'.kibana':
privileges: indices:admin/create, indices:admin/exists, indices:admin/mapping/put, indices:admin/mappings/fields/get, indices:admin/refresh, indices:admin/validate/query, indices:data/read/get, indices:data/read/mget, indices:data/read/search, indices:data/write/delete, indices:data/write/index, indices:data/write/update
kibana4_monitoring:
cluster:
- cluster:monitor/nodes/info
- cluster:monitor/health
indices:
'logstash-*':
- indices:admin/mappings/fields/get
- indices:admin/validate/query
- indices:data/read/search
- indices:data/read/msearch
- indices:admin/get
'.kibana':
- indices:admin/create
- indices:admin/exists
- indices:admin/mapping/put
- indices:admin/mappings/fields/get
- indices:admin/refresh
- indices:admin/validate/query
- indices:data/read/get
- indices:data/read/mget
- indices:data/read/search
- indices:data/write/delete
- indices:data/write/index
- indices:data/write/update
# Marvel user role. Assign to marvel users.
marvel_user:
indices:
'.marvel-es-*':
privileges: read
'.kibana':
privileges: indices:admin/exists, indices:admin/mappings/fields/get, indices:admin/validate/query, indices:data/read/get, indices:data/read/mget, indices:data/read/search
# Marvel remote agent role. Assign to the agent user on the remote marvel cluster
# to which the marvel agent will export all its data
remote_marvel_agent:
cluster: indices:admin/template/put, indices:admin/template/get
indices:
'.marvel-es-*':
privileges: all
```
---
### Author:  [@robertranjan](https://discuss.elastic.co/u/robertranjan)
#### Post date: [November 18, 2015, 4:47pm UTC](https://discuss.elastic.co/t/elastic-search-shield-instance-is-not-authenticating-logstash-user/34891/4 "2015-11-18T16:47:24Z")
I did not make any changes to role\_mapping.yml. Here it is:
```
[root@es1 shield]# cat role_mapping.yml
# Role mapping configuration file which has elasticsearch roles as keys
# that map to one or more user or group distinguished names
#roleA: this is an elasticsearch role
# - groupA-DN this is a group distinguished name
# - groupB-DN
# - user1-DN this is the full user distinguished name
#power_user:
# - "cn=admins,dc=example,dc=com"
#user:
# - "cn=users,dc=example,dc=com"
# - "cn=admins,dc=example,dc=com"
# - "cn=John Doe,cn=other users,dc=example,dc=com"
```
---
### Author:  [@robertranjan](https://discuss.elastic.co/u/robertranjan)
#### Post date: [November 18, 2015, 6:52pm UTC](https://discuss.elastic.co/t/elastic-search-shield-instance-is-not-authenticating-logstash-user/34891/5 "2015-11-18T18:52:47Z")
logstash elasticsearch output:
```
output {
elasticsearch {
#protocol => "transport"
#hosts => ["es1.DOMAIN:9200","es2.DOMAIN:9200","es3.DOMAIN:9200"]
#hosts => ["es1.DOMAIN","es2.DOMAIN","es3.DOMAIN"]
#index => "logstash-%{+YYYY.MM.dd}"
#port => "9300"
hosts => ["es1.DOMAIN:9200"]
keystore => "/opt/keys/cert.pem"
keystore_password => "plain_text_password_with_@_symbol"
user => "logstash"
password => "logstash" # removed symbols for now
#password => "hash_copied_from_es_users"
#network_host => "es1.DOMAIN"
}
}
```
---
### Author:  [@jaymode](https://discuss.elastic.co/u/jaymode)
#### Post date: [November 19, 2015, 9:40pm UTC](https://discuss.elastic.co/t/elastic-search-shield-instance-is-not-authenticating-logstash-user/34891/6 "2015-11-19T21:40:36Z")
can you print the roles of the logstash user using the esusers command? `esusers list logstash`
---
### Author:  [@robertranjan](https://discuss.elastic.co/u/robertranjan)
#### Post date: [November 23, 2015, 9:03pm UTC](https://discuss.elastic.co/t/elastic-search-shield-instance-is-not-authenticating-logstash-user/34891/7 "2015-11-23T21:03:03Z")
> [@jaymode](#):
>
> esusers list logstash
I'm sorry for the delayed response.
```
[root@es1 elasticsearch]# bin/shield/esusers list logstash
logstash : logstash
[root@es1 elasticsearch]#
```
---
### Author:  [@pickypg](https://discuss.elastic.co/u/pickypg)
#### Post date: [November 25, 2015, 1:39am UTC](https://discuss.elastic.co/t/elastic-search-shield-instance-is-not-authenticating-logstash-user/34891/8 "2015-11-25T01:39:13Z")
Maybe we can take a step back and explore something simpler. I just freshly installed Shield to attempt this by creating a Logstash user without _anything_ else added or modified (beyond setting the `network.host` setting):
1. Install Elasticsearch 2.0.0
2. Install Shield and License plugins
3. Create Logstash user with the existing `logstash` role:
```auto
```
bin/shield/esusers logstash -p logstash -r logstash
```
1. Start Elasticsearch.
2. Create a JSON file to send as part of a manual `_bulk` payload, which I will call `test.json` (note: I picked an index name that does not exist so that it can be safely deleted afterward). **The `\n`'s (press enter) are meant to be literally typed by you and it's important to note that both lines must end with one!**
```json
```
{ "index" : { "\_index" : "logstash-dne", "\_type" : "log", "\_id" : "1" } }\n
{ "message" : "this is text!" }\n
```
1. Use the above file to create the index via the `_bulk` API:
```auto
```
$ curl -u logstash -XPOST es1.DOMAIN:9200/\_bulk --data-binary @test.json
```
On my side, this worked. Once you can verify that this works, then we can probably start to work backward to figure out what is different so that we can determine why it's not working.
Note: I did not add any keystore settings anywhere because I did not configure SSL/TLS.
---
### Author:  [@robertranjan](https://discuss.elastic.co/u/robertranjan)
#### Post date: [December 7, 2015, 9:56pm UTC](https://discuss.elastic.co/t/elastic-search-shield-instance-is-not-authenticating-logstash-user/34891/9 "2015-12-07T21:56:10Z")
I don't know what was the problem but re-installing ES and shield fixed my issue.
Thanks for your help. You may go ahead and close this ticket as RESOLVED.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:47pm UTC](https://discuss.elastic.co/t/elastic-search-shield-instance-is-not-authenticating-logstash-user/34891/10 "2017-07-06T13:47:33Z")