# AmazonClientException\[Unable to unmarshall error response...\] when creating snapshot in S3 **URL:** \ **Category:** Elasticsearch\ **Created:** [April 25, 2014, 10:14pm UTC](https://discuss.elastic.co/t/amazonclientexception-unable-to-unmarshall-error-response-when-creating-snapshot-in-s3/17211 "2014-04-25T22:14:38Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Paulo\_Correa](https://avatars.discourse-cdn.com/v4/letter/p/bb73d2/32.png) [@Paulo\_Correa](https://discuss.elastic.co/u/Paulo_Correa)\ **Post date:** [April 25, 2014, 10:14pm UTC](https://discuss.elastic.co/t/amazonclientexception-unable-to-unmarshall-error-response-when-creating-snapshot-in-s3/17211/1 "2014-04-25T22:14:38Z")
I`ve set up ES v.1.1.1 + AWS-cloud-plugin 2.1.1 on an EC2 instance, using a Role to give access to my S3 bucket. Everything seems fine, I can put, list and get objects from inside the instance (tested with node.js code), but when I try to create an ES snapshot I get the following error message: _$ curl -XPUT "localhost:9200/\_snapshot/my\_snapshot\_repo/snapshot1?wait\_for\_completion=true"_ {"error":"SnapshotCreationException[[my\_snapshot\_repo:snapshot1] failed to create snapshot]; nested: IOException[Failed to get [snapshot-snapshot1]]; nested: AmazonClientException[Unable to unmarshall error response (The declaration for the entity "ContentType" must end with '\>'.). Response Code: 417, Response Text: Expectation Failed]; nested: SAXParseException[The declaration for the entity "ContentType" must end with '\>'.]; ","status":500} _Full error stack here_: [https://gist.github.com/anonymous/11304863](https://gist.github.com/anonymous/11304863) _I`ve set up my snapshot repository with:_ $ curl -XPUT '[http://localhost:9200/\_snapshot/my\_snapshot\_repo](http://localhost:9200/_snapshot/my_snapshot_repo)' -d '{ "type": "s3", "settings": { "bucket": "mybucket","region": "sa-east"}}' My _elasticsearch.yml_ aws config is as follows: ... cluster.name: montadores cloud.aws.region: sa-east-1 discovery: type: ec2 discovery.ec2.tag.elasticsearch\_cluster: mobile\_montadores cloud.node.auto\_attributes: true ... _Discovery seems OK_, as I only have 1 instance running: ... [2014-04-25 21:16:53,265][INFO][cluster.service] [Cap 'N Hawk] new\_master [Cap 'N Hawk][M013xeCxQSu8QbZzdNucVA][ip-10-152-89-168][inet[/10.152.89.168:9300]]{aws\_availability\_zone=sa-east-1a}, reason: zen-disco-join (elected\_as\_master) ... [2014-04-25 21:16:56,005][INFO][node] [Cap 'N Hawk] started The only similar thing I found to this error was an old closed issue: [https://github.com/elasticsearch/elasticsearch/issues/1137](https://github.com/elasticsearch/elasticsearch/issues/1137) Can anybody shed some light if I`m doing something wrong? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/112aa619-e4db-4dc7-9a30-c86e8c0d2414%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/112aa619-e4db-4dc7-9a30-c86e8c0d2414%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Pete\_Michel](https://avatars.discourse-cdn.com/v4/letter/p/6f9a4e/32.png) [@Pete\_Michel](https://discuss.elastic.co/u/Pete_Michel)\ **Post date:** [May 8, 2014, 1:21pm UTC](https://discuss.elastic.co/t/amazonclientexception-unable-to-unmarshall-error-response-when-creating-snapshot-in-s3/17211/2 "2014-05-08T13:21:48Z")
Paulo, Did you ever figure out your error? I just encountered the exact same problem and was hoping you had found a solution Thanks, Pete On Friday, April 25, 2014 6:14:38 PM UTC-4, Paulo Correa wrote: > I`ve set up ES v.1.1.1 + AWS-cloud-plugin 2.1.1 on an EC2 instance, using > a Role to give access to my S3 bucket. Everything seems fine, I can put, > list and get objects from inside the instance (tested with node.js code), > but when I try to create an ES snapshot I get the following error message: > > _$ curl -XPUT > "localhost:9200/\_snapshot/my\_snapshot\_repo/snapshot1?wait\_for\_completion=true"_ > {"error":"SnapshotCreationException[[my\_snapshot\_repo:snapshot1] failed to > create snapshot]; nested: IOException[Failed to get [snapshot-snapshot1]]; > nested: AmazonClientException[Unable to unmarshall error response (The > declaration for the entity "ContentType" must end with '\>'.). Response > Code: 417, Response Text: Expectation Failed]; nested: > SAXParseException[The declaration for the entity "ContentType" must end > with '\>'.]; ","status":500} > > _Full error stack here_: [elasticsearch log error · GitHub](https://gist.github.com/anonymous/11304863) > > _I`ve set up my snapshot repository with:_ > > $ curl -XPUT '[http://localhost:9200/\_snapshot/my\_snapshot\_repo](http://localhost:9200/_snapshot/my_snapshot_repo)' -d '{ > "type": "s3", "settings": { "bucket": "mybucket","region": "sa-east"}}' > > My _elasticsearch.yml_ aws config is as follows: > > ... > cluster.name: montadores > cloud.aws.region: sa-east-1 > discovery: > type: ec2 > > discovery.ec2.tag.elasticsearch\_cluster: mobile\_montadores > cloud.node.auto\_attributes: true > ... > > _Discovery seems OK_, as I only have 1 instance running: > > ... > > [2014-04-25 21:16:53,265][INFO][cluster.service] [Cap 'N Hawk] > new\_master [Cap 'N > Hawk][M013xeCxQSu8QbZzdNucVA][ip-10-152-89-168][inet[/10.152.89.168:9300]]{aws\_availability\_zone=sa-east-1a}, > reason: zen-disco-join (elected\_as\_master) > > ... > > [2014-04-25 21:16:56,005][INFO][node] [Cap 'N Hawk] > started > > The only similar thing I found to this error was an old closed issue: > [Snapshotting to S3 occasionally fails · Issue #1137 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1137) > > Can anybody shed some light if I`m doing something wrong? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f86f89fc-6517-4e1b-ae6b-bce9e93d24e7%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f86f89fc-6517-4e1b-ae6b-bce9e93d24e7%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Paulo\_Correa](https://avatars.discourse-cdn.com/v4/letter/p/bb73d2/32.png) [@Paulo\_Correa](https://discuss.elastic.co/u/Paulo_Correa)\ **Post date:** [May 8, 2014, 1:51pm UTC](https://discuss.elastic.co/t/amazonclientexception-unable-to-unmarshall-error-response-when-creating-snapshot-in-s3/17211/3 "2014-05-08T13:51:28Z")
Hi Pete, we ran some more tests to see if we could narrow down the problem. We found out that whenever we had an instance that was not in AWS' default-vpc launch an snapshot, the problem occurred. That was the only factor that made the problem happen, so it is not an issue with permissions, role, buckets, etc, but rather may be a routing problem. We're thinking of submitting a ticket to AWS Support, as we're still facing the error when we launch an instance in our internal VPC. Em quinta-feira, 8 de maio de 2014 10h21min48s UTC-3, Pete Michel escreveu: > Paulo, > > Did you ever figure out your error? I just encountered the exact same > problem and was hoping you had found a solution > > Thanks, > Pete > > On Friday, April 25, 2014 6:14:38 PM UTC-4, Paulo Correa wrote: > > > I`ve set up ES v.1.1.1 + AWS-cloud-plugin 2.1.1 on an EC2 instance, > > using a Role to give access to my S3 bucket. Everything seems fine, I can > > put, list and get objects from inside the instance (tested with node.js > > code), but when I try to create an ES snapshot I get the following error > > message: > > > > _$ curl -XPUT > > "localhost:9200/\_snapshot/my\_snapshot\_repo/snapshot1?wait\_for\_completion=true"_ > > {"error":"SnapshotCreationException[[my\_snapshot\_repo:snapshot1] failed > > to create snapshot]; nested: IOException[Failed to get > > [snapshot-snapshot1]]; nested: AmazonClientException[Unable to unmarshall > > error response (The declaration for the entity "ContentType" must end > > with '\>'.). Response Code: 417, Response Text: Expectation Failed]; nested: > > SAXParseException[The declaration for the entity "ContentType" must end > > with '\>'.]; ","status":500} > > > > _Full error stack here_: [elasticsearch log error · GitHub](https://gist.github.com/anonymous/11304863) > > > > _I`ve set up my snapshot repository with:_ > > > > $ curl -XPUT '[http://localhost:9200/\_snapshot/my\_snapshot\_repo](http://localhost:9200/_snapshot/my_snapshot_repo)' -d '{ > > "type": "s3", "settings": { "bucket": "mybucket","region": "sa-east"}}' > > > > My _elasticsearch.yml_ aws config is as follows: > > > > ... > > cluster.name: montadores > > cloud.aws.region: sa-east-1 > > discovery: > > type: ec2 > > > > discovery.ec2.tag.elasticsearch\_cluster: mobile\_montadores > > cloud.node.auto\_attributes: true > > ... > > > > _Discovery seems OK_, as I only have 1 instance running: > > > > ... > > > > [2014-04-25 21:16:53,265][INFO][cluster.service] [Cap 'N Hawk] > > new\_master [Cap 'N > > Hawk][M013xeCxQSu8QbZzdNucVA][ip-10-152-89-168][inet[/10.152.89.168:9300]]{aws\_availability\_zone=sa-east-1a}, > > reason: zen-disco-join (elected\_as\_master) > > > > ... > > > > [2014-04-25 21:16:56,005][INFO][node] [Cap 'N Hawk] > > started > > > > The only similar thing I found to this error was an old closed issue: > > [Snapshotting to S3 occasionally fails · Issue #1137 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1137) > > > > Can anybody shed some light if I`m doing something wrong? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/261e366b-232e-4474-ba47-4d01c18af569%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/261e366b-232e-4474-ba47-4d01c18af569%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Pete\_Michel](https://avatars.discourse-cdn.com/v4/letter/p/6f9a4e/32.png) [@Pete\_Michel](https://discuss.elastic.co/u/Pete_Michel)\ **Post date:** [May 8, 2014, 3:54pm UTC](https://discuss.elastic.co/t/amazonclientexception-unable-to-unmarshall-error-response-when-creating-snapshot-in-s3/17211/4 "2014-05-08T15:54:01Z")
I was able to resolve my issue. We have elasticsearch in a private subnet. Access to S3 goes through squid proxy. I was able to alter the squid proxy configuration to fix the issue. I added the following to /etc/squid/squid.conf... ignore\_error\_100 on For reference, here is the squid proxy docs ([squid : ignore\_expect\_100 configuration directive](http://www.squid-cache.org/Doc/config/ignore_expect_100/)) Not sure if you are in the same situation, but hope this helps. --Pete On Thursday, May 8, 2014 9:51:28 AM UTC-4, Paulo Correa wrote: > Hi Pete, > > we ran some more tests to see if we could narrow down the problem. We > found out that whenever we had an instance that was not in AWS' default-vpc > launch an snapshot, the problem occurred. That was the only factor that > made the problem happen, so it is not an issue with permissions, role, > buckets, etc, but rather may be a routing problem. > > We're thinking of submitting a ticket to AWS Support, as we're still > facing the error when we launch an instance in our internal VPC. > > Em quinta-feira, 8 de maio de 2014 10h21min48s UTC-3, Pete Michel escreveu: > > > Paulo, > > > > Did you ever figure out your error? I just encountered the exact same > > problem and was hoping you had found a solution > > > > Thanks, > > Pete > > > > On Friday, April 25, 2014 6:14:38 PM UTC-4, Paulo Correa wrote: > > > > > I`ve set up ES v.1.1.1 + AWS-cloud-plugin 2.1.1 on an EC2 instance, > > > using a Role to give access to my S3 bucket. Everything seems fine, I can > > > put, list and get objects from inside the instance (tested with node.js > > > code), but when I try to create an ES snapshot I get the following error > > > message: > > > > > > _$ curl -XPUT > > > "localhost:9200/\_snapshot/my\_snapshot\_repo/snapshot1?wait\_for\_completion=true"_ > > > {"error":"SnapshotCreationException[[my\_snapshot\_repo:snapshot1] failed > > > to create snapshot]; nested: IOException[Failed to get > > > [snapshot-snapshot1]]; nested: AmazonClientException[Unable to unmarshall > > > error response (The declaration for the entity "ContentType" must end > > > with '\>'.). Response Code: 417, Response Text: Expectation Failed]; nested: > > > SAXParseException[The declaration for the entity "ContentType" must end > > > with '\>'.]; ","status":500} > > > > > > _Full error stack here_: [elasticsearch log error · GitHub](https://gist.github.com/anonymous/11304863) > > > > > > _I`ve set up my snapshot repository with:_ > > > > > > $ curl -XPUT '[http://localhost:9200/\_snapshot/my\_snapshot\_repo](http://localhost:9200/_snapshot/my_snapshot_repo)' -d '{ > > > "type": "s3", "settings": { "bucket": "mybucket","region": "sa-east"}}' > > > > > > My _elasticsearch.yml_ aws config is as follows: > > > > > > ... > > > cluster.name: montadores > > > cloud.aws.region: sa-east-1 > > > discovery: > > > type: ec2 > > > > > > discovery.ec2.tag.elasticsearch\_cluster: mobile\_montadores > > > cloud.node.auto\_attributes: true > > > ... > > > > > > _Discovery seems OK_, as I only have 1 instance running: > > > > > > ... > > > > > > [2014-04-25 21:16:53,265][INFO][cluster.service] [Cap 'N > > > Hawk] new\_master [Cap 'N > > > Hawk][M013xeCxQSu8QbZzdNucVA][ip-10-152-89-168][inet[/10.152.89.168:9300]]{aws\_availability\_zone=sa-east-1a}, > > > reason: zen-disco-join (elected\_as\_master) > > > > > > ... > > > > > > [2014-04-25 21:16:56,005][INFO][node] [Cap 'N > > > Hawk] started > > > > > > The only similar thing I found to this error was an old closed issue: > > > [Snapshotting to S3 occasionally fails · Issue #1137 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1137) > > > > > > Can anybody shed some light if I`m doing something wrong? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/477b6020-911b-406a-b37e-a27543fe1c7e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/477b6020-911b-406a-b37e-a27543fe1c7e%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:30am UTC](https://discuss.elastic.co/t/amazonclientexception-unable-to-unmarshall-error-response-when-creating-snapshot-in-s3/17211/5 "2017-07-06T01:30:46Z")