S3 - setting and verifying repo issue

What about even more basic for example?

PUT _snapshot/my_s3_repository
{
  "type": "s3",
  "settings": {
    "bucket": "elasticsearch.mydomain.com-snapshots-eu-central-1"
  }
}

If you see from my setup, it connects via https:

[2021-10-14T18:28:29,658][DEBUG][c.a.request              ] [IsaacAsimov] Sending Request: GET https://elasticsearch.mydomain.com-snapshots-eu-central-1.s3.eu-central-1.amazonaws.com /index-5 Headers: (amz-sdk-invocation-id: 476dd792-df59-31fd-c147-2f5d83c0d0a2, Content-Type: application/octet-stream, User-Agent: aws-sdk-java/1.11.749 Linux/4.19.0-18-amd64 OpenJDK_64-Bit_Server_VM/16.0.2+7 java/16.0.2 vendor/Eclipse_Foundation, ) 
[2021-10-14T18:28:29,659][DEBUG][c.a.a.AWS4Signer         ] [IsaacAsimov] AWS4 Canonical Request: '"GET
/index-5

Thanks guys, I did engage elastic support couple weeks ago but because it's not a Sev 1 the response time is usually much slower than what's needed esp. when am trying things. This forum is surely much quicker to respond and so thought of trying it out here.

@zx8086: FYI - this is an in-house S3 compatible storage (not AWS cloud) and so am using what our system admins provided me to connect to it. It works when tried using aws cli from the same elastic node.

What is the aws cli code you are using ?

I would imagine without http setting you can see a true error if it rejects the plain traffic.

@vee

Interesting......

I am testing 7.13.2 Single node on my localhost

I added the plugin

./bin/elasticsearch-plugin install repository-s3

I just did this with the literal values to see

PUT _snapshot/my_s3_repository
{
  "type": "s3",
  "settings": {
    "bucket": "my-bucket",
    "endpoint" : "s3host",
    "protocol" : "http"
  }
}

In my logs I see this!


[2021-10-14T09:52:28,131][DEBUG][c.a.h.IdleConnectionReaper] [ceres] Reaper thread: 
java.lang.InterruptedException: sleep interrupted
        at java.lang.Thread.sleep(Native Method) ~[?:?]
        at com.amazonaws.http.IdleConnectionReaper.run(IdleConnectionReaper.java:188) [aws-java-sdk-core-1.11.749.jar:?]
[2021-10-14T09:52:28,132][DEBUG][c.a.h.IdleConnectionReaper] [ceres] Shutting down reaper thread.
[2021-10-14T09:52:28,145][INFO ][o.e.r.RepositoriesService] [ceres] updated repository [my_s3_repository]
[2021-10-14T09:52:28,146][DEBUG][o.e.r.s.S3Service        ] [ceres] Using instance profile credentials
[2021-10-14T09:52:28,146][DEBUG][o.e.r.s.S3Service        ] [ceres] using endpoint [http://s3host] and region [null] <!----- NOTE Correct Endpoint 

[ceres] using endpoint [http://s3host] and region [null] <!----- NOTE Correct Endpoint not with the bucket prefix

Perhaps you should uninstall and re-install the s3 plugin...

Or try version 7.13.2 perhaps there was a bug

Interesting, I am infact using v7.13.2 on my end (my bad, I just mentioned the minor version earlier). Interesting that when you tried to connect - you don't see the bucket names prefix. Did you had to enable any additional permissions listed out at the time of plugin installation?

bin]$ ./elasticsearch-plugin install repository-s3
-> Installing repository-s3
-> Downloading repository-s3 from elastic
[=================================================] 100%  
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.SocketPermission * connect,resolve
* java.util.PropertyPermission es.allow_insecure_settings read,write
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed repository-s3
-> Please restart Elasticsearch to activate any plugins installed

Nope my plugin install looks exactly the same as yours....

except I do notice something really minor, I run mine from the elastic root directory

./bin/elasticsearch-plugin install repository-s3

sometime the path to that root directory is used to set some config etc. (probably not... but we are looking for anything at this point)

What JDK? The Default?

@vee It worked for without and then later on adding those permissions to the java.policy file

@Stephen - yes, default JDK: install-dir/jdk. That's what both JAVA_HOME and ES_JAVA_HOME are pointing to. Tried to remove and re-install plugin from elastic root dir like you - still same behavior.

@zx8086 - didn't quite follow you - did you mean you had to add the java.policy file with those listed permissions while installing plugin? If so, can you share that java.policy file and it's path?

Hi @vee

I mean, i was able to run it fine without those permissions. I then added the permissions and it still worked as before, so the permissions did not alter my set-up in anyway.

/etc/java-11-openjdk/security/java.policy

I am running Debian Buster 10

Linux 4.19.0-18-amd64 #1 SMP Debian 4.19.208-1 (2021-09-29) x86_64 GNU/Linux

Could you please share the java.policy you used? May be I will give that a try. I see a default.policy at /jdk/lib/security/ but not sure if it covers all the required permissions.

@vee

Here... but i feel your issue might be with the setup of your private cloud

/
// This system policy file grants a set of default permissions to all domains
// and can be configured to grant additional permissions to modules and other
// code sources. The code source URL scheme for modules linked into a
// run-time image is "jrt".
//
// For example, to grant permission to read the "foo" property to the module
// "com.greetings", the grant entry is:
//
// grant codeBase "jrt:/com.greetings" {
//     permission java.util.PropertyPermission "foo", "read";
// };
//

// default permissions granted to all domains
grant {
    // allows anyone to listen on dynamic ports
    permission java.net.SocketPermission "localhost:0", "listen";

    // "standard" properies that can be read by anyone
    permission java.util.PropertyPermission "java.version", "read";
    permission java.util.PropertyPermission "java.vendor", "read";
    permission java.util.PropertyPermission "java.vendor.url", "read";
    permission java.util.PropertyPermission "java.class.version", "read";
    permission java.util.PropertyPermission "os.name", "read";
    permission java.util.PropertyPermission "os.version", "read";
    permission java.util.PropertyPermission "os.arch", "read";
    permission java.util.PropertyPermission "file.separator", "read";
    permission java.util.PropertyPermission "path.separator", "read";
    permission java.util.PropertyPermission "line.separator", "read";
    permission java.util.PropertyPermission
                   "java.specification.version", "read";
    permission java.util.PropertyPermission "java.specification.vendor", "read";
    permission java.util.PropertyPermission "java.specification.name", "read";
    permission java.util.PropertyPermission
                   "java.vm.specification.version", "read";
    permission java.util.PropertyPermission
                   "java.vm.specification.vendor", "read";
    permission java.util.PropertyPermission
                   "java.vm.specification.name", "read";
    permission java.util.PropertyPermission "java.vm.version", "read";
    permission java.util.PropertyPermission "java.vm.vendor", "read";
    permission java.util.PropertyPermission "java.vm.name", "read";

    // 
    permission java.lang.RuntimePermission accessDeclaredMembers
    permission java.lang.RuntimePermission getClassLoader
    permission java.lang.reflect.ReflectPermission suppressAccessChecks
    permission java.net.SocketPermission * connect,resolve
    permission java.util.PropertyPermission es.allow_insecure_settings read,write

};

@vee How did you install tar.gz deb? rpm?

There is something not right with the environment or something ...

Any chance you can just pull down an Elasticsearch 7.13.2 tar.gz , untar it , change no settings , start it and try to apply the same repository command... and see what we get?

PUT _snapshot/my_s3_repository
{
  "type": "s3",
  "settings": {
    "bucket": "my-bucket",
    "endpoint" : "s3host",
    "protocol" : "http"
  }
}
curl -X PUT "localhost:9200/_snapshot/my_s3_repository?pretty" -H 'Content-Type: application/json' -d'
{
  "type": "s3",
  "settings": {
    "bucket": "my-bucket",
    "endpoint" : "s3host",
    "protocol" : "http"
  }
}
'

@stephenb : We installed it by downloading the artifact and unzipping it (tar). Not through a package manager. We use ansible to rollout to all the environments, and so the same has been working across all environments. with no other issues so far. Although let me try from a different node and see if I can do a fresh installation.

@zx8086 : Tried to use same values in java.policy as part of the default.policy file but no luck :frowning:

//
// Permissions required by modules stored in a run-time image and loaded
// by the platform class loader.
//
// NOTE that this file is not intended to be modified. If additional
// permissions need to be granted to the modules in this file, it is
// recommended that they be configured in a separate policy file or
// ${java.home}/conf/security/java.policy.
//


grant codeBase "jrt:/java.compiler" {
    permission java.security.AllPermission;
};


grant codeBase "jrt:/java.net.http" {
    permission java.lang.RuntimePermission "accessClassInPackage.sun.net";
    permission java.lang.RuntimePermission "accessClassInPackage.sun.net.util";
    permission java.lang.RuntimePermission "accessClassInPackage.sun.net.www";
    permission java.lang.RuntimePermission "accessClassInPackage.jdk.internal.misc";
    permission java.net.SocketPermission "*","connect,resolve";
    permission java.net.URLPermission "http:*","*:*";
    permission java.net.URLPermission "https:*","*:*";
    permission java.net.URLPermission "ws:*","*:*";
    permission java.net.URLPermission "wss:*","*:*";
    permission java.net.URLPermission "socket:*","CONNECT";  // proxy
    // For request/response body processors, fromFile, asFile
    permission java.io.FilePermission "<<ALL FILES>>","read,write,delete";
    permission java.util.PropertyPermission "*","read";
    permission java.net.NetPermission "getProxySelector";
};

grant codeBase "jrt:/java.scripting" {
    permission java.security.AllPermission;
};

grant codeBase "jrt:/java.security.jgss" {
    permission java.security.AllPermission;
};

grant codeBase "jrt:/java.smartcardio" {
    permission javax.smartcardio.CardPermission "*", "*";
    permission java.lang.RuntimePermission "loadLibrary.j2pcsc";
    permission java.lang.RuntimePermission
                   "accessClassInPackage.sun.security.jca";
    permission java.lang.RuntimePermission
                   "accessClassInPackage.sun.security.util";
    permission java.util.PropertyPermission
                   "javax.smartcardio.TerminalFactory.DefaultType", "read";
    permission java.util.PropertyPermission "os.name", "read";
    permission java.util.PropertyPermission "os.arch", "read";
    permission java.util.PropertyPermission "sun.arch.data.model", "read";
    permission java.util.PropertyPermission
                   "sun.security.smartcardio.library", "read";
    permission java.util.PropertyPermission
                   "sun.security.smartcardio.t0GetResponse", "read";
    permission java.util.PropertyPermission
                   "sun.security.smartcardio.t1GetResponse", "read";
    permission java.util.PropertyPermission
                   "sun.security.smartcardio.t1StripLe", "read";
    // needed for looking up native PC/SC library
    permission java.io.FilePermission "<<ALL FILES>>","read";
    permission java.security.SecurityPermission "putProviderProperty.SunPCSC";
    permission java.security.SecurityPermission
                   "clearProviderProperties.SunPCSC";
    permission java.security.SecurityPermission
                   "removeProviderProperty.SunPCSC";
};

grant codeBase "jrt:/java.sql" {
    permission java.security.AllPermission;
};

grant codeBase "jrt:/java.sql.rowset" {
    permission java.security.AllPermission;
};


grant codeBase "jrt:/java.xml.crypto" {
    permission java.lang.RuntimePermission
                   "getStackWalkerWithClassReference";
    permission java.lang.RuntimePermission
                   "accessClassInPackage.sun.security.util";
    permission java.util.PropertyPermission "*", "read";
    permission java.security.SecurityPermission "putProviderProperty.XMLDSig";
    permission java.security.SecurityPermission
                   "clearProviderProperties.XMLDSig";
    permission java.security.SecurityPermission
                   "removeProviderProperty.XMLDSig";
    permission java.security.SecurityPermission
                   "com.sun.org.apache.xml.internal.security.register";
    permission java.security.SecurityPermission
                   "getProperty.jdk.xml.dsig.secureValidationPolicy";
    permission java.lang.RuntimePermission
                   "accessClassInPackage.com.sun.org.apache.xml.internal.*";
    permission java.lang.RuntimePermission
                   "accessClassInPackage.com.sun.org.apache.xpath.internal";
    permission java.lang.RuntimePermission
                   "accessClassInPackage.com.sun.org.apache.xpath.internal.*";
};


grant codeBase "jrt:/jdk.accessibility" {
    permission java.lang.RuntimePermission "accessClassInPackage.sun.awt";
};

grant codeBase "jrt:/jdk.charsets" {
    permission java.util.PropertyPermission "os.name", "read";
    permission java.lang.RuntimePermission "charsetProvider";
    permission java.lang.RuntimePermission
                   "accessClassInPackage.jdk.internal.misc";
    permission java.lang.RuntimePermission "accessClassInPackage.sun.nio.cs";
};

grant codeBase "jrt:/jdk.crypto.ec" {
    permission java.lang.RuntimePermission
                   "accessClassInPackage.sun.security.*";
    permission java.lang.RuntimePermission "loadLibrary.sunec";
    permission java.security.SecurityPermission "putProviderProperty.SunEC";
    permission java.security.SecurityPermission "clearProviderProperties.SunEC";
    permission java.security.SecurityPermission "removeProviderProperty.SunEC";
};

grant codeBase "jrt:/jdk.crypto.cryptoki" {
    permission java.lang.RuntimePermission
                   "accessClassInPackage.sun.security.*";
    permission java.lang.RuntimePermission "accessClassInPackage.sun.nio.ch";
    permission java.lang.RuntimePermission "loadLibrary.j2pkcs11";
    permission java.util.PropertyPermission "sun.security.pkcs11.allowSingleThreadedModules", "read";
    permission java.util.PropertyPermission "sun.security.pkcs11.disableKeyExtraction", "read";
    permission java.util.PropertyPermission "os.name", "read";
    permission java.util.PropertyPermission "os.arch", "read";
    permission java.util.PropertyPermission "jdk.crypto.KeyAgreement.legacyKDF", "read";
    permission java.security.SecurityPermission "putProviderProperty.*";
    permission java.security.SecurityPermission "clearProviderProperties.*";
    permission java.security.SecurityPermission "removeProviderProperty.*";
    permission java.security.SecurityPermission
                   "getProperty.auth.login.defaultCallbackHandler";
    permission java.security.SecurityPermission "authProvider.*";
    // Needed for reading PKCS11 config file and NSS library check
    permission java.io.FilePermission "<<ALL FILES>>", "read";
};

grant codeBase "jrt:/jdk.dynalink" {
    permission java.security.AllPermission;
};

grant codeBase "jrt:/jdk.httpserver" {
    permission java.security.AllPermission;
};

grant codeBase "jrt:/jdk.internal.le" {
    permission java.security.AllPermission;
};

grant codeBase "jrt:/jdk.internal.vm.compiler" {
    permission java.security.AllPermission;
};

grant codeBase "jrt:/jdk.internal.vm.compiler.management" {
    permission java.lang.RuntimePermission "accessClassInPackage.jdk.internal.vm.compiler.collections";
    permission java.lang.RuntimePermission "accessClassInPackage.jdk.vm.ci.runtime";
    permission java.lang.RuntimePermission "accessClassInPackage.jdk.vm.ci.services";
    permission java.lang.RuntimePermission "accessClassInPackage.org.graalvm.compiler.core.common";
    permission java.lang.RuntimePermission "accessClassInPackage.org.graalvm.compiler.debug";
    permission java.lang.RuntimePermission "accessClassInPackage.org.graalvm.compiler.hotspot";
    permission java.lang.RuntimePermission "accessClassInPackage.org.graalvm.compiler.options";
    permission java.lang.RuntimePermission "accessClassInPackage.org.graalvm.compiler.phases.common.jmx";
    permission java.lang.RuntimePermission "accessClassInPackage.org.graalvm.compiler.serviceprovider";
};

grant codeBase "jrt:/jdk.jsobject" {
    permission java.security.AllPermission;
};

grant codeBase "jrt:/jdk.localedata" {
    permission java.lang.RuntimePermission "accessClassInPackage.sun.text.*";
    permission java.lang.RuntimePermission "accessClassInPackage.sun.util.*";
};

grant codeBase "jrt:/jdk.naming.dns" {
    permission java.security.AllPermission;
};

grant codeBase "jrt:/jdk.scripting.nashorn" {
    permission java.security.AllPermission;
};

grant codeBase "jrt:/jdk.scripting.nashorn.shell" {
    permission java.security.AllPermission;
};

grant codeBase "jrt:/jdk.security.auth" {
    permission java.security.AllPermission;
};

grant codeBase "jrt:/jdk.security.jgss" {
    permission java.security.AllPermission;
};

grant codeBase "jrt:/jdk.zipfs" {
    permission java.io.FilePermission "<<ALL FILES>>", "read,write,delete";
    permission java.lang.RuntimePermission "fileSystemProvider";
    permission java.lang.RuntimePermission "accessUserInformation";
    permission java.util.PropertyPermission "os.name", "read";
    permission java.util.PropertyPermission "user.dir", "read";
    permission java.util.PropertyPermission "user.name", "read";
};

// permissions needed by applications using java.desktop module
grant {
    permission java.lang.RuntimePermission "accessClassInPackage.com.sun.beans";
    permission java.lang.RuntimePermission "accessClassInPackage.com.sun.beans.*";
    permission java.lang.RuntimePermission "accessClassInPackage.com.sun.java.swing.plaf.*";
    permission java.lang.RuntimePermission "accessClassInPackage.com.apple.*";
};

grant {
    // allows anyone to listen on dynamic ports
    permission java.net.SocketPermission "localhost:0", "listen";

    // "standard" properies that can be read by anyone
    permission java.util.PropertyPermission "java.version", "read";
    permission java.util.PropertyPermission "java.vendor", "read";
    permission java.util.PropertyPermission "java.vendor.url", "read";
    permission java.util.PropertyPermission "java.class.version", "read";
    permission java.util.PropertyPermission "os.name", "read";
    permission java.util.PropertyPermission "os.version", "read";
    permission java.util.PropertyPermission "os.arch", "read";
    permission java.util.PropertyPermission "file.separator", "read";
    permission java.util.PropertyPermission "path.separator", "read";
    permission java.util.PropertyPermission "line.separator", "read";
    permission java.util.PropertyPermission
                   "java.specification.version", "read";
    permission java.util.PropertyPermission "java.specification.vendor", "read";
    permission java.util.PropertyPermission "java.specification.name", "read";
    permission java.util.PropertyPermission
                   "java.vm.specification.version", "read";
    permission java.util.PropertyPermission
                   "java.vm.specification.vendor", "read";
    permission java.util.PropertyPermission
                   "java.vm.specification.name", "read";
    permission java.util.PropertyPermission "java.vm.version", "read";
    permission java.util.PropertyPermission "java.vm.vendor", "read";
    permission java.util.PropertyPermission "java.vm.name", "read";

    //
    permission java.lang.RuntimePermission "accessDeclaredMembers";
    permission java.lang.RuntimePermission "getClassLoader";
    permission java.lang.reflect.ReflectPermission "suppressAccessChecks";
    permission java.net.SocketPermission "*", "connect,resolve";
    permission java.util.PropertyPermission "es.allow_insecure_settings", "read,write";

};

If it is still pre-pending the bucket name to the URL none the rest matters I think....

Tried to install ES on a new VM, downloaded v7.13.2, untar and updated the Elasticsearch.yml and other required config files. I see some warning messages with bucket prefix on the hostname.

Elasticsearch.yml:

bootstrap.memory_lock: true
cluster.name: test-cluster
cluster.remote.connect: false
discovery.type: single-node
http.port: 9200
node.data: true
node.ingest: true
node.master: true
transport.port: 9300
node.name: ingest1
path.data: <path-to-data>/ingest1/data
path.logs: <path-to-logs>

network.host: hostname
network.publish_host: hostname
network.bind_host: hostname

xpack.security.enabled: false
xpack.security.transport.ssl.enabled: false
xpack.security.http.ssl.enabled: false
xpack.security.audit.enabled: true
xpack.security.audit.logfile.events.exclude: "access_granted"

Here are all the warnings that I see when I run the create snapshot command:

PUT _snapshot/my_s3_repository
{
  "type": "s3",
  "settings": {
    "bucket": "bucket1",
    "endpoint": "host:port",
    "protocol": "http"
  }
}

Enabled trace and here are the key line entries from stack trace:

101 [2021-10-15T13:01:16,493][WARN ][c.a.j.SdkMBeanRegistrySupport] [ingest1]
102 java.security.AccessControlException: access denied ("javax.management.MBeanServerPermission" "findMBeanServer")
.
.
138 [2021-10-15T13:01:16,500][WARN ][c.a.m.AwsSdkMetrics      ] [ingest1]
139 java.security.AccessControlException: access denied ("javax.management.MBeanServerPermission" "findMBeanServer")
.
.
175 [2021-10-15T13:01:16,515][WARN ][c.a.s.s.i.UseArnRegionResolver] [ingest1] Unable to load config file null
176 java.security.AccessControlException: access denied ("java.io.FilePermission" "/home/rabbitmq/.aws/config" "read")
.
.
215 [2021-10-15T13:01:16,606][DEBUG][c.a.m.CsmConfigurationProviderChain] [ingest1] Unable to load configuration from com.amazonaws.monitoring.EnvironmentVariableCsmConfigurationProv    ider@66a74051: Unable to load Client Side Monitoring configurations from environment variables!
216 [2021-10-15T13:01:16,606][DEBUG][c.a.m.CsmConfigurationProviderChain] [ingest1] Unable to load configuration from com.amazonaws.monitoring.SystemPropertyCsmConfigurationProvider@    42e4e92a: Unable to load Client Side Monitoring configurations from system properties variables!
217 [2021-10-15T13:01:16,608][DEBUG][c.a.m.CsmConfigurationProviderChain] [ingest1] Unable to load configuration from com.amazonaws.monitoring.ProfileCsmConfigurationProvider@60edb19    0: Unable to load config file
.
.
241 [2021-10-15T13:01:16,749][DEBUG][c.a.h.c.ClientConnectionManagerFactory] [ingest1]
242 java.lang.reflect.InvocationTargetException: null

.
.
302 [2021-10-15T13:01:16,754][DEBUG][c.a.request              ] [ingest1] Retrying Request: PUT http://bucket1.host:port /tests-uxc-LcwNTbWVZbvm1CdNng/master.dat     Headers: (amz-sdk-invocation-id: 4f63df5f-dd24-597a-3844-7bdab523e8e6, Content-Length: 22, Content-Type: application/octet-stream, User-Agent: aws-sdk-java/1.11.749 Linux/4.18.0-305    .10.2.el8_4.x86_64 OpenJDK_64-Bit_Server_VM/16+36 java/16 vendor/AdoptOpenJDK, x-amz-acl: private, x-amz-storage-class: STANDARD, )
303 [2021-10-15T13:01:16,755][DEBUG][c.a.h.AmazonHttpClient   ] [ingest1] Retriable error detected, will retry in 77ms, attempt number: 0
304 [2021-10-15T13:01:16,833][DEBUG][c.a.a.AWS4Signer         ] [ingest1] AWS4 Canonical Request: '"PUT
.
.
325 [2021-10-15T13:01:16,835][DEBUG][c.a.h.c.ClientConnectionManagerFactory] [ingest1]
326 java.lang.reflect.InvocationTargetException: null
.
.
373 Caused by: java.net.UnknownHostException: bucket1.host
.
.
548 org.elasticsearch.repositories.RepositoryVerificationException: [my_s3_repository] path  is not accessible on master node
549 Caused by: java.io.IOException: Unable to upload object [tests-uxc-LcwNTbWVZbvm1CdNng/master.dat] using a single upload

Paste bin: Full stack trace
https://pastebin.pl/view/cdbeee05

Thanks for your time!!

Perhaps try

"path_style_access" : true
PUT _snapshot/my_s3_repository
{
  "type": "s3",
  "settings": {
    "bucket": "my-bucket",
    "endpoint" : "hostname:5500",
    "protocol" : "http",
    "path_style_access" : true
  }
}

https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-bucket-intro.html

Wohoo!! It finally worked on the stand alone node after adding the property you shared @stephenb . Thanks so much for that. I was able to connect using https endpoint as well just fine from the test node. I did try this property earlier but the value true was listed as string and not boolean and so it didn't work.

"path_style_access" : "true"
1 Like

Darn ... should have seen that earlier!
But good to know... now we are all experts :wink:

Absolutely! I'm glad it's resolved after much struggle. Truly appreciate the help I got from this forum here!

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.