# \[RESOLVED\] Sporadically failing snapshots

**URL:** <https://discuss.elastic.co/t/resolved-sporadically-failing-snapshots/144645>\
**Category:** Elasticsearch\
**Created:** [August 16, 2018, 7:48am UTC](https://discuss.elastic.co/t/resolved-sporadically-failing-snapshots/144645 "2018-08-16T07:48:27Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![vicpav](https://avatars.discourse-cdn.com/v4/letter/v/9de053/32.png) [@vicpav](https://discuss.elastic.co/u/vicpav)\
**Post date:** [August 16, 2018, 7:48am UTC](https://discuss.elastic.co/t/resolved-sporadically-failing-snapshots/144645/1 "2018-08-16T07:48:28Z")

</div>

Hi,

We are using Elasticsearch 6.3.x deployed to Elastic Cloud which is configured against GCS (Europe West 1). About a week ago, some of the snapshots started to fail for 1-2 indices (out of ~500) with the warning in the log similar to that:

> com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113) ~[?:?] at  
> com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40) ~[?:?] at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432) ~[?:?] at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352) ~[?:?] at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469) ~[?:?] at org.elasticsearch.repositories.gcs.GoogleCloudStorageBlobStore.lambda$writeBlob$5(GoogleCloudStorageBlobStore.java:213) ~[?:?] at org.elasticsearch.repositories.gcs.SocketAccess.lambda$doPrivilegedVoidIOException$0(SocketAccess.java:54) ~[?:?] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0\_144] at org.elasticsearch.repositories.gcs.SocketAccess.doPrivilegedVoidIOException(SocketAccess.java:53) ~[?:?] at org.elasticsearch.repositories.gcs.GoogleCloudStorageBlobStore.writeBlob(GoogleCloudStorageBlobStore.java:207) ~[?:?] at org.elasticsearch.repositories.gcs.GoogleCloudStorageBlobContainer.writeBlob(GoogleCloudStorageBlobContainer.java:72) ~[?:?] at org.elasticsearch.repositories.blobstore.BlobStoreRepository$SnapshotContext.snapshotFile(BlobStoreRepository.java:1281) ~[elasticsearch-6.3.2.jar:6.3.2] at org.elasticsearch.repositories.blobstore.BlobStoreRepository$SnapshotContext.snapshot(BlobStoreRepository.java:1217) ~[elasticsearch-6.3.2.jar:6.3.2] ... 9 more

The issue affects both 6.3.1 and 6.3.2. Does anybody else see the same issue?

Any idea what to do about it is very much welcome.

---

<div class="post-metadata">

**Author:** ![vicpav](https://avatars.discourse-cdn.com/v4/letter/v/9de053/32.png) [@vicpav](https://discuss.elastic.co/u/vicpav)\
**Post date:** [August 16, 2018, 12:54pm UTC](https://discuss.elastic.co/t/resolved-sporadically-failing-snapshots/144645/2 "2018-08-16T12:54:54Z")

</div>

OK, looks like I am not the only one and Elastic Cloud is looking at it, see [Elastic Cloud Status - Incident "Increased cluster snapshot failures on GCP regions"](https://found.statuspage.io/incidents/k8nfvb2sy8n7)

---

<div class="post-metadata">

**Author:** ![vicpav](https://avatars.discourse-cdn.com/v4/letter/v/9de053/32.png) [@vicpav](https://discuss.elastic.co/u/vicpav)\
**Post date:** [August 17, 2018, 5:31am UTC](https://discuss.elastic.co/t/resolved-sporadically-failing-snapshots/144645/3 "2018-08-17T05:31:51Z")

</div>

The issue seems to be fixed around midnight (UTC) on August 17, 2018.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 14, 2018, 5:35am UTC](https://discuss.elastic.co/t/resolved-sporadically-failing-snapshots/144645/4 "2018-09-14T05:35:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
