# Logstash won't write to ES with logstash role **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-security\ **Created:** [November 9, 2016, 11:21pm UTC](https://discuss.elastic.co/t/logstash-wont-write-to-es-with-logstash-role/65567 "2016-11-09T23:21:33Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![stecino](https://avatars.discourse-cdn.com/v4/letter/s/ea666f/32.png) [@stecino](https://discuss.elastic.co/u/stecino)\ **Post date:** [November 9, 2016, 11:21pm UTC](https://discuss.elastic.co/t/logstash-wont-write-to-es-with-logstash-role/65567/1 "2016-11-09T23:21:33Z")
I have read all some of the posts that pertain to this subject, but still couldn't make it work. Getting this error, when trying to write some random generated data to ES Failed action. {:status=\>403, :action=\>["index", {:\_id=\>nil, :\_index=\>"oganes-2016.11.09", :\_type=\>"logs", :\_routing=\>nil}, #\, @cancelled=false, @data={"message"=\>"line 3", "@version"=\>"1", "@timestamp"=\>"2016-11-09T23:14:20.259Z", "host"=\>"[herd-blah.com](http://herd-blah.com)", "sequence"=\>2}, @metadata={}, @accessors=#\"line 3", "@version"=\>"1", "@timestamp"=\>"2016-11-09T23:14:20.259Z", "host"=\>"[herd-blah.com](http://herd-blah.com)", "sequence"=\>2}, @lut={"host"=\>[{"message"=\>"line 3", "@version"=\>"1", "@timestamp"=\>"2016-11-09T23:14:20.259Z", "host"=\>"[herd-blah.com](http://herd-blah.com)", "sequence"=\>2}, "host"], "sequence"=\>[{"message"=\>"line 3", "@version"=\>"1", "@timestamp"=\>"2016-11-09T23:14:20.259Z", "host"=\>"[herd-blah.com](http://herd-blah.com)", "sequence"=\>2}, "sequence"], "type"=\>[{"message"=\>"line 3", "@version"=\>"1", "@timestamp"=\>"2016-11-09T23:14:20.259Z", "host"=\>"[herd-blah.com](http://herd-blah.com)", "sequence"=\>2}, "type"]}\>\>], :response=\>{"index"=\>{"\_index"=\>"oganes-2016.11.09", "\_type"=\>"logs", "\_id"=\>nil, "status"=\>403, "error"=\>{"type"=\>"security\_exception", "reason"=\>"action [indices:admin/create] is unauthorized for user [lbviewer]"}}}, :level=\>:warn} my roles.yml admin: cluster: - all indices: - names: '\*' privileges: - all # The required role for logstash users logstash: cluster: - manage\_index\_templates indices: - names: 'oganes\*' privileges: - all my role\_mapping.yml logstash: - "CN=lbviewer,OU=Service Accounts,OU=System Accounts,DC=corp,DC=blah,DC=com" if I move this under admin: mapping then it works fine, but I want to be able to run it as logstash role What am I doing wrong? ---
**Author:** ![jaymode](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaymode/32/50103_2.png) [@jaymode](https://discuss.elastic.co/u/jaymode)\ **Post date:** [November 10, 2016, 2:03pm UTC](https://discuss.elastic.co/t/logstash-wont-write-to-es-with-logstash-role/65567/2 "2016-11-10T14:03:41Z")
Have you checked for errors about parsing the files in the elasticsearch log file? ---
**Author:** ![stecino](https://avatars.discourse-cdn.com/v4/letter/s/ea666f/32.png) [@stecino](https://discuss.elastic.co/u/stecino)\ **Post date:** [November 10, 2016, 11:30pm UTC](https://discuss.elastic.co/t/logstash-wont-write-to-es-with-logstash-role/65567/3 "2016-11-10T23:30:59Z")
I doubt if the parsing error happens, since when I move the user from admin mapping to logstash, my shield trace shows that user is authenticated as [logstash]. Shouldn't parsing error be logged in my elastic data nodes? Another test I did was with authenticate, with user in different role mappings. curl -XGET -u lbviewer [http://herd-es1:9201/\_shield/authenticate](http://herd-es1:9201/_shield/authenticate) Enter host password for user 'lbviewer': {"username":"lbviewer","roles":["admin"],"full\_name":null,"email":null,"metadata":{}} curl -XGET -u lbviewer [http://herd-es1:9201/\_shield/authenticate](http://herd-es1:9201/_shield/authenticate) Enter host password for user 'lbviewer': {"username":"lbviewer","roles":["logstash"],"full\_name":null,"email":null,"metadata":{}} did you want me validate roles.yml? ---
**Author:** ![stecino](https://avatars.discourse-cdn.com/v4/letter/s/ea666f/32.png) [@stecino](https://discuss.elastic.co/u/stecino)\ **Post date:** [November 10, 2016, 11:45pm UTC](https://discuss.elastic.co/t/logstash-wont-write-to-es-with-logstash-role/65567/4 "2016-11-10T23:45:12Z")
I ran yml validate ruby command ruby -e "require 'yaml';puts YAML.load\_file('./roles.yml')" logstashclustermanage\_index\_templatesindicesnamesoganes_privilegesallkibana4\_serverclustermonitorindicesnames.kibana_privilegesallpower\_userclustermonitorindicesnames_privilegesalltransport\_clientclustertransport\_clientremote\_marvel\_agentclustermanage\_index\_templatesindicesnames.marvel-es-privilegesalluserindicesnamesprivilegesreadadminclusterallindicesnames_privilegesallmarvel\_userindicesnames.marvel-es-\*privilegesreadnames.kibanaprivilegesview\_index\_metadataread ruby -e "require 'yaml';puts YAML.load\_file('./role\_mapping.yml')" logstashCN=lbviewer,OU=Service Accounts,OU=System Accounts,DC=corp,DC=blah,DC=comkibana4\_serverCN=KibanaServer,OU=Service Accounts,OU=System Accounts,DC=corp,DC=blah,DC=comadminCN=SiteReliability,OU=Roles,OU=Groups,DC=corp,DC=blah,DC=com seems to be ok ---
**Author:** ![jaymode](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaymode/32/50103_2.png) [@jaymode](https://discuss.elastic.co/u/jaymode)\ **Post date:** [November 14, 2016, 4:04pm UTC](https://discuss.elastic.co/t/logstash-wont-write-to-es-with-logstash-role/65567/5 "2016-11-14T16:04:31Z")
yes a parsing error will be logged. Can you set the log level to debug and see what group DNs are returned from LDAP or AD? ---
**Author:** ![stecino](https://avatars.discourse-cdn.com/v4/letter/s/ea666f/32.png) [@stecino](https://discuss.elastic.co/u/stecino)\ **Post date:** [November 14, 2016, 8:02pm UTC](https://discuss.elastic.co/t/logstash-wont-write-to-es-with-logstash-role/65567/6 "2016-11-14T20:02:18Z")
So here is what I am seeing in the ES logs [2016-11-14 19:57:50,004][DEBUG][shield.authc.support] [herd-es1-blue] the roles [[]], are mapped from these [ldap] groups [[CN=lbviewers,OU=Security,OU=Groups,DC=corp,DC=blah,DC=com]] for realm [ldap/ldap1] [2016-11-14 19:57:50,004][DEBUG][shield.authc.support] [herd-es1-blue] the roles [[logstash]], are mapped from the user [ldap] for realm [CN=lbviewer,OU=Service Accounts,OU=System Accounts,DC=corp,DC=blah,DC=com/ldap] [2016-11-14 19:57:50,014][DEBUG][shield.authc.ldap] [herd-es1-blue] authenticated user [lbviewer], with roles [[logstash]] [2016-11-14 19:57:50,249][DEBUG][shield.authc.ldap] [herd-es1-blue] authenticated user [lbviewer], with roles [[logstash]] [2016-11-14 19:57:50,377][DEBUG][shield.authc.ldap] [herd-es1-blue] authenticated user [lbviewer], with roles [[logstash]] [2016-11-14 19:57:50,409][DEBUG][shield.authc.ldap] [herd-es1-blue] authenticated user [lbviewer], with roles [[logstash]] [2016-11-14 19:57:50,674][DEBUG][shield.authc.ldap] [herd-es1-blue] authenticated user [lbviewer], with roles [[logstash]] Looks like the group is assigned empty role, but in my case I am using a user that's part of the group and it's roles are properly assigned. Does the group override the user? ---
**Author:** ![jaymode](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaymode/32/50103_2.png) [@jaymode](https://discuss.elastic.co/u/jaymode)\ **Post date:** [November 14, 2016, 8:17pm UTC](https://discuss.elastic.co/t/logstash-wont-write-to-es-with-logstash-role/65567/7 "2016-11-14T20:17:43Z")
That looks good to me. Do you have the role on all of the nodes? Is it the same everywhere? Are you using any roles created via the API or only file based roles? ---
**Author:** ![stecino](https://avatars.discourse-cdn.com/v4/letter/s/ea666f/32.png) [@stecino](https://discuss.elastic.co/u/stecino)\ **Post date:** [November 14, 2016, 10:24pm UTC](https://discuss.elastic.co/t/logstash-wont-write-to-es-with-logstash-role/65567/8 "2016-11-14T22:24:27Z")
It should be, at least on the data nodes. I will check the Master nodes and query nodes. I have assigned native realm order of 1, but based on what users and roles I have setup, non of that applies and won't be in conflict: This is what initially setup when I was messing with local access. curl -XGET -u \*\*\*\*\* '[http://localhost:9201/\_shield/role?pretty](http://localhost:9201/_shield/role?pretty)' { "kibana4\_server\_role" : { "cluster" : ["all"], "indices" : [ { "names" : [".kibana\*"], "privileges" : ["all"] } ], "run\_as" : [] }, "kibana4\_server" : { "cluster" : ["all"], "indices" : [ { "names" : ["\*"], "privileges" : ["indices:data/read/search"] } ], "run\_as" : ["test"] } curl -XGET -u \*\*\*\* '[http://localhost:9201/\_shield/user?pretty](http://localhost:9201/_shield/user?pretty)' { "kibana-server" : { "username" : "kibana-server", "roles" : ["kibana4\_server"], "full\_name" : null, "email" : null, "metadata" : { } }, "kibana\_server" : { "username" : "kibana\_server", "roles" : ["kibana4\_server"], "full\_name" : null, "email" : null, "metadata" : { } } } ---
**Author:** ![stecino](https://avatars.discourse-cdn.com/v4/letter/s/ea666f/32.png) [@stecino](https://discuss.elastic.co/u/stecino)\ **Post date:** [November 14, 2016, 10:53pm UTC](https://discuss.elastic.co/t/logstash-wont-write-to-es-with-logstash-role/65567/9 "2016-11-14T22:53:47Z")
WORKED!!! So my Master nodes were missing update roles.yml and roles\_mapping.yml file Now logstash can create the index. Thanks alot for helping and pointing me to the right direction. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 12, 2016, 10:53pm UTC](https://discuss.elastic.co/t/logstash-wont-write-to-es-with-logstash-role/65567/10 "2016-12-12T22:53:50Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.