Can't change replica count for '.security' index

Elastic Stack Elasticsearch
1

We're running 9 instances of Elasticsearch 2.4.0 across 3 hosts (yes, 3 instances per host - don't ask), and we've got Shield installed.

When Shield was installed, (I assume) it created a '.security' index for native auth. It seems the plugin installer looked at the number of nodes in the cluster and set the number of replicas to be n-1, i.e. 8, to ensure that every node has a copy of '.security'.

The issue is that because of allocation awareness, the cluster won't allocate multiple replicas of this index on the same host (which makes perfect sense for all our indices, so we're not going to change it), hence the cluster shows as yellow because there are 5 replicas that can't be assigned (primary on one instance on the first host, a replica on an instance on each of the other two hosts, and five replicas going nowhere) and it leaves our cluster in a yellow state.

Obvious solution then is to change the number of replicas for this index from 8 to 2. Secondary solution is to delete the '.security' index entirely, because we use file and PKI realms only. However, I can't seem to do anything to get sufficient access rights to complete either of those tasks:

curl -k -u es_admin -XPUT https://$(hostname --fqdn):9200/.security/_settings -d '{ "number_of_replicas": 2 }'

{
  "error" : {
    "root_cause" : [ {
      "type" : "security_exception",
      "reason" : "action [indices:admin/settings/update] is unauthorized for user [es_admin]"
    } ],
    "type" : "security_exception",
    "reason" : "action [indices:admin/settings/update] is unauthorized for user [es_admin]"
  },
  "status" : 403
}

Here's my Shield configuration files:

/etc/elasticsearch/shield/roles.yml

admin:
  cluster:
    - all
  indices:
    - names:
        - '*'
      privileges:
        - all

/etc/elasticsearch/shield/users_roles

admin:es_admin,marvel_agent

Having read https://www.elastic.co/guide/en/shield/current/limitations.html#_accessing_the_literal_security_literal_index, I've tried all variations that I can think of in roles.yml, including specifying the index name and cluster 'manage_security' privilege explicitly, but nothing seems to grant me enough access to change the index.

This is a non-prod environment, hence it's sat in the backlog for ages, but it would be nice to get the cluster green so that we can re-enable our alerting.

Anyone come across this?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.