Upgrade to Elastic 8.0.0 (Windows Zip package)

Elastic Stack Elasticsearch
1

Error

“… will not overwrite keystore …, because this incurs changing the file owner”

Verbose

at org.elasticsearch.common.settings.KeyStoreWrapper.save(KeyStoreWrapper.java:608)
at org.elasticsearch.common.settings.KeyStoreWrapper.save(KeyStoreWrapper.java:540)
at org.elasticsearch.common.settings.KeyStoreWrapper.upgrade(KeyStoreWrapper.java:339)
at org.elasticsearch.common.settings.KeyStoreWrapper.bootstrap(KeyStoreWrapper.java:225)
at org.elasticsearch.bootstrap.BootstrapUtil.loadSecureSettings(BootstrapUtil.java:60)
at org.elasticsearch.bootstrap.BootstrapUtil.loadSecureSettings(BootstrapUtil.java:55)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:301)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:157)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:77)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112)
at org.elasticsearch.cli.Command.main(Command.java:77)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:122)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80)

Quick Notes

  • Hosting Elastic on-premises on Server 2019
  • Current version 7.16.1 installed using the MSI Installer.
  • First upgrade to version 7.17.0 with the Zip package.
  • Version 7.17.0 upgraded successfully and installed as Windows services using “Elasticsearch-service.bat” instead of the previous “Elasticsearch.exe”.
  • The Migration Assistant did not flag any warnings or major issues.
  • The windows service is running as “Local System” account.
  • The owner of the keystore file is “System”, match the account used by the windows service.
  • Generate the same error when running the “Elasticsearch.bat” or “Elasticsearch-keystore upgrade”. Expected since I am not running as “System” account.

Advises needed

  • How to resolve this error?
  • Is this a bug or am I missing something, as version 7.17.0 can read the keystore file without any problem?
  • I was pondering about re-creating the keystore but the concern is that it will generate a different keystore.seed and therefore not able to read the existing data and unless it is the last resort, I really prefer not to perform snapshot recovery.

Any helps and advises will be appreciated! Thanks!

2 Likes
2

Updates

Platform: Windows 2019
This is an existing Elastic Instance.

Managed to resolve this error by providing an encrypted password for the key-store. In my defend, this feature wasn’t available when we first started. Unfortunately, another one pop up.

I can start Elastic directly via the Elasticsearch.bat command but it asked for the key-store password (btw it is display as clear text on the console).

Use the Elasticsearch-service.bat to create a windows service and verified that the settings are correct. In both the Window Services and the Command console, it has no errors in both the elastic logs as well as the java.io.tmpdir. The Elastic Instance did not start up. I can’t stop the services with the Elasticsearch-service.bat stop command, I must kill the process manually.

It seen to be missing the password for the key-store, how do I set this parameter, pass the this information or am I missing something?

Any suggestions? Thanks!

3

I was having the same issue with my elastic upgrade and the solution was to change the owner of file Elasticsearch.keystore to both Elasticsearch:Elasticsearch owner.
Well at least that is for linux, I'm sure it'll be similar for windows.

4

Thanks lusynda!

I did not work for me. I have tried various combinations. From setting the services to run and assigned ownership to the keystore file as Networks Service to a proper Domain System Account. It made no differences.

Something that my colleague said, “When the Null Hypothesis cannot be proven then the Alternative Hypothesis must be true”.

Is it possible that “elasticsearch-service-x64” is not working properly?

From the following Observations

bin\elasticsearch -V
Version: 8.1.0, Build: default/zip/3700f7679f7d95e36da0b43762189bab189bc53a/2022-03-03T14:20:00.690422633Z, JVM: 17.0.2

bin\elasticsearch-service-x64.exe

Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: java.lang.IllegalStateException: Keystore passphrase required but none provided.
Likely root cause: java.lang.IllegalStateException: Keystore passphrase required but none provided.
at org.elasticsearch.bootstrap.BootstrapUtil.readPassphrase(BootstrapUtil.java:48)
at org.elasticsearch.bootstrap.BootstrapUtil.lambda$loadSecureSettings$0(BootstrapUtil.java:60)
at org.elasticsearch.common.settings.KeyStoreWrapper.bootstrap(KeyStoreWrapper.java:213)
at org.elasticsearch.bootstrap.BootstrapUtil.loadSecureSettings(BootstrapUtil.java:60)
at org.elasticsearch.bootstrap.BootstrapUtil.loadSecureSettings(BootstrapUtil.java:55)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:301)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:157)
at org.elasticsearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:81)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112)
at org.elasticsearch.cli.Command.main(Command.java:77)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:122)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80)

Peeking at the “elasticsearch-service.bat”, I cannot find the option to provide this, nor any indications that argument can be passed to “elasticsearch-service-x64.exe”. Is there a place to specify this via “elasticsearch-service.bat manager”.
Am I really missing something?

Thanks!

5

Have you tried to add the keystore passphrase like the log said.
The passphrase that maybe your transport tls or http tls require.

6

The way i solve my issue was to move the keystore file to the other location then i restart elastic, after that i saw what elastic require of the file keystore so i change the permission of the old keystore file then i move the old file back in then restart elastic again.
Maybe you should try to do that, it could show you some other form of error that maybe easier to fix.

7

@lusynda In my case, ES is running as docker container and the keys to be added to keystore are mounted as volumes. They are added to keystore during ES startup, which essentially gives the same error as reported by @TonyLuc initially.
Is there a way to resolve it on the runtime ?

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.