Run two versions of ES on machine

This is on a W10 box. ES (7.10.2) is currently running on localhost:9200. I need to upgrade. When I attempted to upgrade to 7.16.3 some time ago a regression occurred, reported by me and acknowledged by Elasticsearch HQ, which hopefully will have been corrected in the current version (8.6.2). But I absolutely need to do some testing to ensure that my stuff won't break, hence two versions, one on another localhost port.

I have just downloaded the latest stable release, 8.6.2 and unzipped the .zip.

I want 8.6.2 to run as a W10 service, for example at localhost:9500.

I don't need any warnings about "the danger" of doing this. All this is on a single PC, and when I'm happy that 8.6.2 doesn't break any tests I'll switch to that.

I found this recent answer on the same subject (*nix OS).

So (having unzipped 8.6.2) I changed all those settings in elasticsearch.yml.

I ran elasticsearch.bat.

Error:

[2023-03-04T15:59:45,029][ERROR][o.e.b.Elasticsearch      ] [M17A] fatal exception while booting Elasticsearch
java.lang.RuntimeException: org.elasticsearch.cli.UserException: will not overwrite keystore at [D:\apps\ElasticSearch\elasticsearch-7.10.2\config\elasticsearch.keystore], because this incurs changing the file owner
	at org.elasticsearch.bootstrap.Elasticsearch.initPhase2(Elasticsearch.java:150) ~[elasticsearch-8.6.2.jar:?]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:66) ~[elasticsearch-8.6.2.jar:?]
Caused by: org.elasticsearch.cli.UserException: will not overwrite keystore at [D:\apps\ElasticSearch\elasticsearch-7.10.2\config\elasticsearch.keystore], because this incurs changing the file owner
	at org.elasticsearch.common.settings.KeyStoreWrapper.save(KeyStoreWrapper.java:480) ~[elasticsearch-8.6.2.jar:?]
	at org.elasticsearch.common.settings.KeyStoreWrapper.save(KeyStoreWrapper.java:412) ~[elasticsearch-8.6.2.jar:?]
	at org.elasticsearch.common.settings.KeyStoreWrapper.upgrade(KeyStoreWrapper.java:291) ~[elasticsearch-8.6.2.jar:?]
	at org.elasticsearch.common.settings.KeyStoreWrapper.bootstrap(KeyStoreWrapper.java:221) ~[elasticsearch-8.6.2.jar:?]
	at org.elasticsearch.bootstrap.Elasticsearch.initPhase2(Elasticsearch.java:148) ~[elasticsearch-8.6.2.jar:?]
	... 1 more

... right: I don't want it to "overwrite keystore at ... \elasticsearch-7.10.2 ...". Why is it using a "key store" from another ES instance? How can I persuade it to make a separate "key store"?

You can not have two instances share data directories, if that is what you are trying to do. If you want to run two nodes on a single host they need to be completely separate.

It might help if you share the configuration files of the two nodes.

Thanks. I confirm that my aim is to have two entirely separate instances of ES running. I want no "sharing of data", indeed no sharing of anything whatsoever. Separate, distinct, apart, unrelated.

The file \config\elasticsearch.yml for 7.10.2 is completely commented-out.

The same file for 8.6.2 has the following uncommented lines:

cluster.name: my-862application
...
node.name: node862-1
...
path.data: D:/temp/es862_data
...
path.logs: D:/temp/es862_logs
...
http.port: 9500

(NB I also tried the above paths with Windows backslashes in the paths: same error).

... are there other "configuration files" which I need to modify for the 8.6.2 setup?

From the error stack trace above, the culprit line would appear to be KeyStoreWrapper.java, line 480: at this point, for some reason, the 8.6.2. jar file decides it wants to use a path with "7.10.2" in it.

So I checked the source code involved here:

// check that replace doesn't change the owner
if (Files.exists(keystoreFile, LinkOption.NOFOLLOW_LINKS)
	&& false == Files.getOwner(keystoreTempFile, LinkOption.NOFOLLOW_LINKS)
		.equals(Files.getOwner(keystoreFile, LinkOption.NOFOLLOW_LINKS))) {
	String message = String.format(
		Locale.ROOT,
		"will not overwrite keystore at [%s], because this incurs changing the file owner",
		keystoreFile
	);
	throw new UserException(ExitCodes.CONFIG, message);
}

... somehow the variable "keystoreFile" seems to have been set "wrongly".

To me this suggests (possibly) that in the course of "registering" ES something has maybe previously happened like maybe a key has been configured in the W10 registry (or something). But I haven't got sufficient knowledge of this level of gory details.

I did a search in the registry on "7.10.2". The most likely key was this:

Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Elastic\Elasticsearch\7.10.2: several keys, including "CONFIGDIRECTORY: D:\apps\Elasticsearch\elasticsearch-7.10.2\config"

So I created this key:
Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Elastic\Elasticsearch\8.6.2: "CONFIGDIRECTORY: D:\apps\Elasticsearch\elasticsearch-8.6.2\config"

... same error. Needs an expert.

OK, some progress but still not there yet.

After consulting the docs (as usual, not really directed at Windoze users), I did this:

>set ES_PATH_CONF=D:/apps/ElasticSearch/elasticsearch-8.6.2/config
>elasticsearch.bat

... a torrent of output ensued, but it looked like configuring was happening.

When I attempted to browse to localhost:9500, however, I got this output in the console:

[2023-03-05T09:01:48,356][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [node862-1] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/127.0.0.1:9500, remoteAddress=/127.0.0.1:9690}
[2023-03-05T09:01:48,356][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [node862-1] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/127.0.0.1:9500, remoteAddress=/127.0.0.1:9691}
[2023-03-05T09:01:48,356][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [node862-1] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/127.0.0.1:9500, remoteAddress=/127.0.0.1:9692}
[2023-03-05T09:01:48,356][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [node862-1] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/127.0.0.1:9500, remoteAddress=/127.0.0.1:9693}
[2023-03-05T09:01:48,356][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [node862-1] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/127.0.0.1:9500, remoteAddress=/127.0.0.1:9694}
[2023-03-05T09:01:48,371][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [node862-1] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/127.0.0.1:9500, remoteAddress=/127.0.0.1:9695}
[2023-03-05T09:01:48,371][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [node862-1] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/127.0.0.1:9500, remoteAddress=/127.0.0.1:9696}
[2023-03-05T09:01:48,371][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [node862-1] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/127.0.0.1:9500, remoteAddress=/127.0.0.1:9697}
[2023-03-05T09:01:48,371][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [node862-1] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/127.0.0.1:9500, remoteAddress=/127.0.0.1:9698}
[2023-03-05T09:01:48,371][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [node862-1] received plaintext http traffic on an https channel, closing connection Netty4HttpChannel{localAddress=/127.0.0.1:9500, remoteAddress=/127.0.0.1:9699}

... and in the browser page I get a page with an error message: "Connection was reset". No "you know, for search" or other connection details output...

However, I do see that one of the last log messages in the console screen before I tried to get to "localhost:9500" was this:

[2023-03-05T09:01:30,685][INFO ][o.e.c.r.a.AllocationService] [node862-1] current.health="GREEN" message="Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[.geoip_databases][0], [.security-7][0]]])." previous.health="RED" reason="shards started [[.geoip_databases][0], [.security-7][0]]"

which looks hopeful. Seem to be close.

... any diagnosis, suggestions??

later: This is the right answer! ... but then (if using a non-standard port at least) there are security issues which follow. See solution here.

It now seems I am getting into an area which has baffled others:

When I put "https://localhost:9500" in the browser I am asked for a "username" and "password".

It appears that ES 8 more or less requires you to have strong security from the start. Unfortunately the docs here appear to be far from ideal about what to do next. Especially if

  • you're on Windoze
  • you don't know what Docker is, how to install it or use, or what its purpose might be
  • you don't know what Kibana is, how to install it or use it, or what its purpose might be
  • you're not interested in installing Docker or Kibana
  • you're told to set up xpack security, but in fact that all appears to be configured correctly (i.e. to use it) in the yml file.
  • the supposed default pairing username and password "elastic" and "changeme" doesn't get you in
  • there is no listing in the logs showing an initial username and password as suggested might be the case, for example, here.

... however it appears (hopefully) that I have achieved the objective of this question. "http://localhost:7200" still produces the expected output, and my app using 7.10.2 appears to be unaffected by the parallel ES 8.6.2 instance.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.