Superuser unable to manage xpack security

Elastic Stack Elasticsearch
1

I lost the elastic user password since I activated x-pack , I created a new superuser called admin

/bin/x-pack/users list
admin : superuser

now when I access the kibana -> management -> security -> Roles / Users , it shows (logged in as admin)

"You do not have permission to manage roles"

How do I fix this issue ? How do I recover password for elastic user ?

2

This should have worked fine. I assume you have used
bin/x-pack/users useradd admin -r superuser, correct?

Can you share the output of :

curl -u admin 'http://localhost:9200/_xpack/security/_authenticate?pretty'

and

curl -u admin 'http://localhost:9200 ( so that we can see the version you are running ).

Anything interesting in the elasticsearch logs?

3

{
"username" : "admin",
"roles" : [
"superuser"
],
"full_name" : null,
"email" : null,
"metadata" : { },
"enabled" : true
}

curl -u admin 'http://localhost:9200'
{
"name" : "SmUKNKO",
"cluster_name" : "testenv",
"cluster_uuid" : "bw7WrLdSROGdtfWbgiauZQ",
"version" : {
"number" : "6.2.2",
"build_hash" : "10b1edd",
"build_date" : "2018-02-16T19:01:30.685723Z",
"build_snapshot" : false,
"lucene_version" : "7.2.1",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}

4

superuser should be allowed to manage roles obviously. Can you please share any part of your elasticsearch logs that might seem relevant ( i.e. logging in via Kibana and accessing the Management tab ).

If nothing interesting is there you can try enabling DEBUG

PUT /_cluster/settings
{"transient":{"logger.org.elasticsearch.xpack.security.authz": "DEBUG"}}

temporarily while performing the action above.

5

I am not getting any errors in elkcluster_access.log , all lines shows access_granted.
But I am getting these errors in elkcluster.log

[ERROR][o.e.x.s.a.e.ReservedRealm] [SmUKNKO] failed to retrieve password hash for reserved user [elastic]
[INFO ][o.e.x.s.a.AuthenticationService] [SmUKNKO] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[INFO ][o.e.x.w.a.l.ExecutableLoggingAction] [SmUKNKO] Watch [Idle_pct] has exceeded the threshold []
[ERROR][o.e.x.m.c.c.ClusterStatsCollector] [SmUKNKO] collector [cluster_stats] failed to collect data

6

I believe the error you are seeing in Kibana is incorrect.

From what I can see, Kibana will give that error whenever it fails to retrieve the list of available roles from Elasticsearch. So, although it appears to be a permissions problem, it could actually have some other underlying cause.

I believe your cluster (and/or security index) might be suffering from some health issues.
Can you check these:

curl -uadmin 'http://localhost:9200/_cluster/health?pretty' 

curl -uadmin 'http://localhost:9200/.security-6' --head 

curl -uadmin 'http://localhost:9200/_cat/shards'
7

curl -uadmin 'http://localhost:9200/_cluster/health?pretty'

{
"cluster_name" : "elk-cluster",
"status" : "yellow",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 1556,
"active_shards" : 1556,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 1541,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 50.24216984178237
}

curl -uadmin 'http://localhost:9200/.security-6' --head
HTTP/1.1 404 Not Found
content-type: application/json; charset=UTF-8
content-length: 367

8

How do I recover the elastic user password ?
I keep getting the above mentioned error logs on elastic user .

9

It doesn't look like the Elastic user has a password yet.
Have you tried running setup-passwords?

10

I tried, I am getting these errors.

x-pack/setup-passwords interactive

Failed to authenticate user 'elastic' against http://localhost:9200/_xpack/security/_authenticate?pretty
Possible causes include:

  • The password for the 'elastic' user has already been changed on this cluster
  • Your elasticsearch node is running against a different keystore
    This tool used the keystore at /etc/elasticsearch/elasticsearch.keystore

ERROR: Failed to verify bootstrap password

12

I also tried to reset elastic user password using this

curl -u my_admin -XPUT 'http://localhost:9200/_xpack/security/user/elastic/_password?pretty' -H 'Content-Type: application/json' -d'

{ "password": "password" }'

{
"error" : {
"root_cause" : [
{
"type" : "index_not_found_exception",
"reason" : "no such index",
"resource.type" : "index_or_alias",
"resource.id" : ".security-6",
"index_uuid" : "na",
"index" : ".security-6"
}
],
"type" : "index_not_found_exception",
"reason" : "no such index",
"resource.type" : "index_or_alias",
"resource.id" : ".security-6",
"index_uuid" : "na",
"index" : ".security-6"
},
"status" : 404
}

13

I can see an index called .security-v6 but not .security-6.

(sorry about multiple posts, I am not able to edit my previous post )

14

Did you previously run a beta version of Elasticsearch 6.0 on this cluster?

The security index was called .security-v6 during some of the betas.

16

Yes , I used 6.1 before upgrading to the latest version.

I was able to fix the issue using index aliases.

curl -XPOST -u my_admin 'localhost:9200/_aliases?pretty' -H 'Content-Type: application/json' -d'
{
"actions" : [
{ "add" : { "index" : ".security-v6", "alias" : ".security-6" } }
]
}
'

after this I was able to reset my elastic user password and everything is working now.

17

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.