Elasticsearch upgrade 6.3 to 6.8 with security


I tried to upgrade my Elasticsearch cluster (4 nodes) from 6.3 to 6.8 and I failed. In my actual cluster (6.3), I don't have the security enable, in version 6.8 the security is free so I want to enable it. This is my Elasticsearch cluster topology :

  • Node 1 : data/master node
  • Node 2 : data/master node
  • Node 3 : data/master node
  • Node 4 : coordinating only node

This is my upgrade process (based on Elastic documentation) :

  1. Disable shard allocation
  2. Stop indexing and perform a synced flush
  3. Shutdown all nodes
  4. Start each upgraded node
  5. Execute "elasticsearch-setup-passwords" commands to set the passwords for the built-in-users

But, during the step 5 (elasticsearch-setup-passwords) I obtain this following error :

Initiating the setup of passwords for reserved users elastic,apm_system,kibana,logstash_system,beats_system,remote_monitoring_user.
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y
Enter password for [elastic]:
Reenter password for [elastic]:
Reenter password for [remote_monitoring_user]:

Connection failure to: failed: Read timed out

ERROR: Failed to set password for user [apm_system].

And, in my Elasticsearch node logs, I noticed this error :

2019-07-19T12:00:24,051][WARN ][r.suppressed] [elastic_node_1] path: /_xpack/security/user/apm_system/_password, params: {pretty=, username=apm_system}
org.elasticsearch.action.UnavailableShardsException: [.security-6][0] [1] shardIt, [0] active : Timeout waiting for [1m], request: indices:data/write/update

I believe understand the error, maybe because I have disable shard allocation on my cluster and my cluster has yellow state. Why can I fix this problem?

Thank you in advance for your help.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.