Running up Cluster in Docker with TLS on a Mac - Issues

[neil6323 (Neil Short)]

I'm trying to run up an ES cluster which has TLS using Docker on a Mac. I'm using the instructions given in https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html

Versions are all latest available.

MacOS (Intel): 11.5.2
Docker: 4.0.0
Elasticsearch/Kibana: 7.14.1

Step 3 is successful (it says done - but no other output. From memory I did get per-node results when I ran the same process in Linux and Docker).
Step 4 - when I try to bring up the cluster, nodes es02 and es03 start running but then exit.

When I look back at the logs from node ES02 I can see the following which may suggest that step 3 was not successful.

ElasticsearchSecurityException[failed to load SSL configuration [xpack.security.transport.ssl]]; nested: ElasticsearchException[failed to initialize SSL KeyManager - key file [/usr/share/elasticsearch/config/certificates/es02/es02.key] does not exist]; nested: NoSuchFileException[/usr/share/elasticsearch/config/certificates/es02/es02.key];'

I've tried this setup more than once. New files (re-copied from the Elastic site) and new folders and I get the same result.

Also when I ran the process through with Linux I had to change the sysctl -w vm.max_map_count=262144 to complete the process. However, if I follow the instructions for the Mac, to rule this out being an issue, using screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty, I get an error basically that this file is not available. There are some references to this, for example https://github.com/docker/for-mac/issues/4822. Is there still a requirement to amend the vm.max_map_count for a Mac and if so is there another way to achieve this?

Thanks
Neil

[neil6323 (Neil Short)]

An update. I've managed to use the instructions to get past step 4 using version 7.6.2 (Docker and MacOS versions remain the same as before). I believe the issue is with step 3 when certs are created. When it works, there is output related to each host.

Step 3 output when successful using version 7.6.2.

% docker-compose -f create-certs.yml run --rm create_certs

Creating network "es7_6_2_elastic" with driver "bridge"
Creating volume "es7_6_2_certs" with local driver
Pulling create_certs (docker.elastic.co/elasticsearch/elasticsearch:7.6.2)...
7.6.2: Pulling from elasticsearch/elasticsearch
c808caf183b6: Pull complete
d6caf8e15a64: Pull complete
b0ba5f324e82: Pull complete
d7e8c1e99b9a: Pull complete
85c4d6c81438: Pull complete
3119218fac98: Pull complete
914accf214bb: Pull complete
Digest: sha256:59342c577e2b7082b819654d119f42514ddf47f0699c8b54dc1f0150250ce7aa
Status: Downloaded newer image for docker.elastic.co/elasticsearch/elasticsearch:7.6.2
Creating es7_6_2_create_certs_run ... done
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
Archive:  /certs/bundle.zip
   creating: /certs/ca/
  inflating: /certs/ca/ca.crt        
   creating: /certs/es01/
  inflating: /certs/es01/es01.crt    
  inflating: /certs/es01/es01.key    
   creating: /certs/es02/
  inflating: /certs/es02/es02.crt    
  inflating: /certs/es02/es02.key    
   creating: /certs/es03/
  inflating: /certs/es03/es03.crt    
  inflating: /certs/es03/es03.key    
   creating: /certs/kib01/
  inflating: /certs/kib01/kib01.crt  
  inflating: /certs/kib01/kib01.key  

I tried again with 7.14.1 and 8.0.0-alpha and neither work when doing a Docker/Mac deployment. Want to try functionality that needs 7.14 so will stick with Linux Docker standup.