Upgrade to Elasticsearch 6.1 on CentOS 7 fails: cannot create elasticsearch.keystore.tmp

Elastic Stack Elasticsearch
1

I installed the s3 snapshot plugin on my ES 5.6 cluster. Those nodes are CentOS 7.4. I installed ES using yum. I needed to upgrade to v6, so I wanted to make a backup.

I installed the repository-s3. I read the documentation for it (inadvertently, I read v6's docs).

I installed the keystore (elasticsearch-keystore create) and added the s3.client.default.access_key and s3.client.default.secret_key values to it.

I know now that on 5.x, those values should have been on the .yml config, but I made that mistake by not paying attention to the docs version.

Anyway. I left that there, didn't make the backup and tried to do a rolling upgrade to 6.x.

When I restarted the node, it failed to start giving out an error reading like: "could not create /etc/elasticsearch/elasticsearch.keystore.tmp. Check permissions". I do not have the exact text for the error, but that was the gist of it.

I had to rollback to a snapshot of that VM, to get everything to the safe side and forgot to copy the error message. I panicked. Sorry.

Is there a way I can delete the keystore from that install (after removing repository-s3, of course), so I can retry the upgrade to 6?

2

You can just delete the keystore file at /etc/elasticsearch/elasticsearch.keystore and recreate once upgraded. I will see if I can reproduce this upgrade issue.

3

I didn't exactly delete it, but I renamed it to /etc/elasticsearch/elasticsearch.keystore.BAK, after deleting the 2 key/value pairs in it, and restarted the server, to no avail. It still gave the same error. That is my recollection anyway, as I haven't tried the upgrade again.

4

Can you do an ls -l on /etc/elasticsearch? You are running the keystore tool as the elasticsearch user right?

5

ls -lah /etc/elasticsearch/

total 44K
drwxr-s---. 4 root elasticsearch 4.0K Dec 28 06:58 .
drwxr-xr-x. 86 root root 8.0K Jan 14 06:00 ..
-rw-rw----. 1 root elasticsearch 763 Dec 28 04:41 elasticsearch.keystore
-rw-rw----. 1 root elasticsearch 3.3K Feb 20 2017 elasticsearch.yml
-rwxr-x---. 1 root elasticsearch 3.3K Sep 29 2016
elasticsearch.yml.OLD_V_2_4
-rw-rw----. 1 root elasticsearch 2.8K Jan 24 2017
elasticsearch.yml.rpmnew
-rw-rw----. 1 root elasticsearch 3.0K Dec 4 07:51 jvm.options
-rw-rw----. 1 root elasticsearch 4.4K Dec 4 07:51 log4j2.properties
drwxr-x---. 2 root elasticsearch 30 Dec 28 04:34 repository-s3
drwxr-x---. 2 root elasticsearch 6 Dec 4 07:51 scripts

I did not run the elasticsearch-keystore tool as the elasticsearch user.

Anyway, I don't see how that could be relevant, if what I want is to be
able to delete elasticsearch.keystore.

6

@rjernst: So... Do you have any suggestions after the ls -l on /etc/elasticsearh?

7

HI,

Check below links:

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.