Locked Out of Kibana Superuser Can't Modify Index Settings

Elastic Stack Kibana
1

I have a single node ELK instance that I am using for a pilot, and the .kibana_security_session_1 index has index.blocks.read_only_allow_delete set to true due to running low on disk space. I have tuned the setting to be more appropriate for my build, but the index is still locked. When I try to flip this to null with a superuser account I get the error "security_exception","reason":"action [indices:admin/settings/update] is unauthorized for user [xxxxxxxx] with roles [superuser] on restricted indices. I don't have access to Kibana to create a new role and map my user account to it that does include the permissions necessary. Is there a way to allow this from the API? I'm fairly comfortable interacting with the node via curl, but I have not been able to find a way to use curl to fix this problem.

2

To add to this, I have attempted to create a role using the API that included access to restricted indices, however sine the kibana security index is the one that is locked, I can't add a role with the permissions necessary to make the change to unlock it. Really unsure where to go from here.

3

As you see from the last line, this index block must be manually released once you have full control of your cluster, with sufficient disc space on all nodes. You can remove the index block with the command listed in the documentation, something like this:

PUT /my_index/_settings
{
"index.blocks.read_only_allow_delete": null
}

Give that a try, also copying @Larry_Gregory for more inputs on this ..they will answer when they get to it .

Thanks
Rashmi

4

Thanks for the response, when I attempt that, I get the below output:

curl --cacert /etc/elasticsearch/certs/http_ca.crt -u maynardbj.sa -X PUT "https://localhost:9200/.kibana_security_session_1/_settings?pretty" -H 'Content-Type: application/json' -d '{ "index.blocks.read_only_allow_delete": null }'
Enter host password for user 'xxxxxxxx':
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "action [indices:admin/settings/update] is unauthorized for user [xxxxxxxx] with roles [superuser] on restricted indices [.kibana_security_session_1], this action is granted by the index privileges [manage,all]"
      }
    ],
    "type" : "security_exception",
    "reason" : "action [indices:admin/settings/update] is unauthorized for user [xxxxxxxx] with roles [superuser] on restricted indices [.kibana_security_session_1], this action is granted by the index privileges [manage,all]"
  },
  "status" : 403
}

When I get the output for the privileges for my 'superuser' role, I get the below output:

{
  "superuser" : {
    "cluster" : [
      "all"
    ],
    "indices" : [
      {
        "names" : [
          "*"
        ],
        "privileges" : [
          "all"
        ],
        "allow_restricted_indices" : false
      },
      {
        "names" : [
          "*"
        ],
        "privileges" : [
          "monitor",
          "read",
          "view_index_metadata",
          "read_cross_cluster"
        ],
        "allow_restricted_indices" : true
      }
    ],
    "applications" : [
      {
        "application" : "*",
        "privileges" : [
          "*"
        ],
        "resources" : [
          "*"
        ]
      }
    ],
    "run_as" : [
      "*"
    ],
    "metadata" : {
      "_reserved" : true
    },
    "transient_metadata" : { }
  }
}

So I have the 'all' privilege on all indices as stated by the error above, but I do not have the allow_restricted_indices set to true for indices. I have tried creating another role with the all and manage and setting this to true, but with the kibana security index locked, it won't allow the creation of a new role.

5

@bjmaynard01 Have you reduced / cleaned up the disk to be below all the thresholds / watermarks?

I ask because I believe recently elastics should be removing those blocks automatically when the disk is below all the watermarks.

2nd is see this in your role

    "allow_restricted_indices" : false

the index you are trying to access is a restricted index.

1 Like
7

You were correct, once I removed enough indexes to get back down below the watermark and restarted everything, I was back in. Thank you.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.