# Snapshot repository AWS S3 bucket via corporate proxy **URL:** \ **Category:** Elastic Cloud Enterprise (ECE)\ **Tags:** snapshot-and-restore\ **Created:** [January 20, 2021, 9:07am UTC](https://discuss.elastic.co/t/snapshot-repository-aws-s3-bucket-via-corporate-proxy/261623 "2021-01-20T09:07:20Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![markn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/markn/32/74417_2.png) [@markn](https://discuss.elastic.co/u/markn)\ **Post date:** [January 20, 2021, 9:07am UTC](https://discuss.elastic.co/t/snapshot-repository-aws-s3-bucket-via-corporate-proxy/261623/1 "2021-01-20T09:07:20Z")
Hi, we run ECE in our own datacenters and internet access is only available via corporate proxy. The AWS S3 bucket works like a charm. ECE runs on RedHat Linux 7. We have configured the corporate proxy in: /usr/lib/systemd/system/docker.service.d/docker.conf > [Unit] > Description=Docker Service > After=multi-user.target > > [Service] > Environment="HTTPS\_PROXY=[http://www-proxy.xyz.com:8080](http://www-proxy.xyz.com:8080)" But it does not work. Error message from Elasticsearch instance: > "reason": "Unable to execute HTTP request: [xyz-repo.s3.amazonaws.com](http://xyz-repo.s3.amazonaws.com)" What is the best practice regarding snapshot repos on AWS, Gcloud or Azure if you run ECE in an on-premise Data Center and the only way to the internet is via a corporate proxy? Thanks. Cheers, Mark P.S. we did not find a clear answer in the documentation or other discussion topics. ---
**Author:** ![Alex\_Piggott](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alex_piggott/32/11053_2.png) [@Alex\_Piggott](https://discuss.elastic.co/u/Alex_Piggott)\ **Post date:** [January 20, 2021, 7:49pm UTC](https://discuss.elastic.co/t/snapshot-repository-aws-s3-bucket-via-corporate-proxy/261623/2 "2021-01-20T19:49:06Z")
Could you tell whether the traffic was reaching the proxy? If you do `docker exec printenv | grep HTTPS` can you tell if `HTTPS_PROXY` is set inside the container? (where `` is `fac--`, you can see the exact names/ids via `docker ps | grep fac`) In terms of best practices, there is a proxy field you can set in the YAML (with `client` set to the repo name i believe - whatever it uses to add the creds to the keystore): [https://www.elastic.co/guide/en/elasticsearch/plugins/master/repository-s3-client.html](https://www.elastic.co/guide/en/elasticsearch/plugins/master/repository-s3-client.html) ---
**Author:** ![markn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/markn/32/74417_2.png) [@markn](https://discuss.elastic.co/u/markn)\ **Post date:** [January 21, 2021, 12:45pm UTC](https://discuss.elastic.co/t/snapshot-repository-aws-s3-bucket-via-corporate-proxy/261623/3 "2021-01-21T12:45:47Z")
Hi Alex, thanks for your response. We are a bit struggling with setting the proxy in de docker containers via /home/$user/.docker/config.json ([https://docs.docker.com/network/proxy/](https://docs.docker.com/network/proxy/)). It seems not to work. Currently work in progress. What we fail to understand is why do we need to set the S3 Repo on ECE level AND we should add the proxy ALSO in the elasticsearch.yml. Because with the latter we would still have manual maintenance on each Elasticsearch cluster (for each ES node) running on our ECE. Would this not contradict with the orchestration principle? Thanks! ---
**Author:** ![Alex\_Piggott](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alex_piggott/32/11053_2.png) [@Alex\_Piggott](https://discuss.elastic.co/u/Alex_Piggott)\ **Post date:** [January 21, 2021, 2:28pm UTC](https://discuss.elastic.co/t/snapshot-repository-aws-s3-bucket-via-corporate-proxy/261623/4 "2021-01-21T14:28:32Z")
The way the Elasticsearch snapshot is structured, you have 3 different elements: - The repository info (_"information needed to snapshot/restore given a blobstore"_, eg the bucket name) ... that lives in `_snapshot` - The client information (_"info the blobstore client needs to connect to the blobstore"_), split into: - insecure settings: blobstore endpoint, proxy settings, etc ... this lives in the YAML - secure settings: creds ... this lives in the keystore For historical reasons the first and third (repo info and secure client info) can be set in the ECE repo object, and ECE then pulls out the creds and puts them in the keystore (Older versions of ES let you put everything together in the `_snapshot` object, but that is no longer the case)/ The insecure settings need to be set separately as part of a plan reconfiguration. The recommended way is that you create a Deployment Template and then those settings are "pre-loaded" into the YAML box when you create a deployment (you'd have to hand add them for existing deployments of course) ---
**Author:** ![markn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/markn/32/74417_2.png) [@markn](https://discuss.elastic.co/u/markn)\ **Post date:** [January 21, 2021, 2:55pm UTC](https://discuss.elastic.co/t/snapshot-repository-aws-s3-bucket-via-corporate-proxy/261623/5 "2021-01-21T14:55:54Z")
Hi Alex, thanks! That makes sense. We will update it accordingly. We already configured the S3 repository at ECE level. So we would set in the elasticsearch.yml (deployment template): s3.client.default.proxy.host: 10.10.10.something s3.client.default.proxy.port: 8080 Just wondering why we would need that proxy information also needs to be configured in the docker container. Would Elasticsearch process not try to go via the proxy directly? Thanks, Mark P.S. the s3.client proxy settings in the YAML did not work (maybe because the proxy env was not set in the Docker containers?). This is the error message from the ECE GUI: ``` There was a problem applying this configuration change Plan change failed: [ClusterFailure:CouldNotEnsureSnapshotRepository]: Unable to ensure existence of snapshot repository. Please validate that the repository is accessible and retry. Details: [{"details":"status: [500 Internal Server Error], entity: [{\"error\":{\"root_cause\": [{\"type\":\"repository_verification_exception\",\"reason\":\"[found-snapshots] path [snapshots/46505578eb2748259fcb664823fd3a5b] is not accessible on master node\"}],\"type\":\"repository_verification_exception\",\"reason\":\"[found-snapshots] path [snapshots/46505578eb2748259fcb664823fd3a5b] is not accessible on master node\",\"caused_by\":{\"type\":\"i_o_exception\",\"reason\":\"Unable to upload object [snapshots/46505578eb2748259fcb664823fd3a5b/tests-qG-rvkiESRuDFO8VF18UfQ/master.dat] using a single upload\",\"caused_by\":{\"type\":\"sdk_client_exception\",\"reason\":\"sdk_client_exception: Unable to execute HTTP request: xyz-repo.s3.amazonaws.com\",\"caused_by\":{\"type\":\"i_o_exception\",\"reason\":\"ece-snapshot-repo.s3.amazonaws.com\"}}}},\"status\":500}], headers: [List()], protocol: [HttpProtocol(HTTP/1.1)]"}] ``` ---
**Author:** ![Alex\_Piggott](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alex_piggott/32/11053_2.png) [@Alex\_Piggott](https://discuss.elastic.co/u/Alex_Piggott)\ **Post date:** [January 21, 2021, 3:04pm UTC](https://discuss.elastic.co/t/snapshot-repository-aws-s3-bucket-via-corporate-proxy/261623/6 "2021-01-21T15:04:59Z")
Unless ES itself doesn't support a host level proxy `*`, I don't see why the approach you initially took wouldn't work. Did you confirm that say `wget` inside the ES docker container was going via the corporate proxy? To my knowledge we've never had anyone configure a proxy (including via Docker), so don't have any experience of ways in which it might fail. I assume just setting the proxy at the host level doesn't work? `*` (I don't know that, the fact it has a proxy field suggests someone had issues at some point though! It would be a question for the stack forum) In terms of your config - is `default` the right `client`? Eg if you do `GET _snapshot/found-snapshots`, is there a `client_name` field? (One other thing I'm not sure about that fails in the "stack question" category is how the proxy setting handles HTTP/HTTPS) ---
**Author:** ![markn](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/markn/32/74417_2.png) [@markn](https://discuss.elastic.co/u/markn)\ **Post date:** [January 21, 2021, 3:25pm UTC](https://discuss.elastic.co/t/snapshot-repository-aws-s3-bucket-via-corporate-proxy/261623/7 "2021-01-21T15:25:58Z")
Hi Alex, thanks. We just got it working by adding the proxy.host and proxy.port in the ECE repository. That seems like an undocumented feature. See screenshot attached. We did not add the proxy information to YAML or docker container. Thanks. Cheers, Mark ! [Screenshot 2021-01-21 at 15.51.01|690x466](https://discuss.elastic.co/uploads/short-url/uj1F8Yq73X6Hre1dqJzzGXScOod.png) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 4, 2021, 3:26pm UTC](https://discuss.elastic.co/t/snapshot-repository-aws-s3-bucket-via-corporate-proxy/261623/8 "2021-02-04T15:26:03Z")
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.