SDK cannot reach AWS S3 repository after upgrading to ES 6.3.0

[Mr.King]

I upgraded from ES 6.2.3 to 6.3.0 and my whole cluster cannot reach my AWS S3 bucket anymore.
Notice that I use the ES docker build.

GET _snapshot/elk-repo/*
{
"error": {
"root_cause": [
{
"type": "repository_missing_exception",
"reason": "[elk-repo] missing"
}
],
"type": "repository_missing_exception",
"reason": "[elk-repo] missing"
},
"status": 404
}

When I want to create my repo:

PUT _snapshot/elk-repo
{
"type": "s3",
"settings":
{
"bucket": "elasticsearch-snapshots",
"base_path": "elk"
}
}

I got the following error:

{
"error": {
"root_cause": [
{
"type": "sdk_client_exception",
"reason": "sdk_client_exception: Unable to execute HTTP request: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"
}
],
"type": "repository_exception",
"reason": "[elk-repo] failed to create repository",
"caused_by": {
"type": "sdk_client_exception",
"reason": "sdk_client_exception: Unable to execute HTTP request: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
"caused_by": {
"type": "i_o_exception",
"reason": "sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
"caused_by": {
"type": "validator_exception",
"reason": "validator_exception: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target",
"caused_by": {
"type": "sun_cert_path_builder_exception",
"reason": "sun_cert_path_builder_exception: unable to find valid certification path to requested target"
}
}
}
}
},
"status": 500
}

I tried to create a brand new container multiple time but I have the same error.

[Mr.King]

It seems this error is related to bad SSL certificates but it is a new install from the docker build. Any idea please?

[tebriel (Chris M)]

I'm also experiencing this with the EC2 Discovery plugin using the 6.3.0 docker.

[2018-07-02T19:59:07,643][INFO ][o.e.d.e.AwsEc2UnicastHostsProvider] [9WyEXt5] Exception while retrieving instance list from AWS API: Unable to execute HTTP request: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

I've reverted to 6.2.4 for now.

[tebriel (Chris M)]

Elasticsearch-Docker Issue #171 mentions this issue and notes that docker.elastic.co/elasticsearch/elasticsearch:6.3.0-cacerts has a fix in it.

[Mr.King]

Thank you tebriel.
I spent hours to find a workaround and finally built my own docker image with the Elasticsearch DEB package.

[tebriel (Chris M)]

Yeah, I tweeted @elastic and they linked me the github issue.