Since upgrading my stack from 5.6.4 to 6.6.0, snapshots to GCS have started failing. The reason given in the error output is:
IndexShardSnapshotFailedException[com.google.cloud.storage.StorageException: Read timed out]; nested: StorageException[Read timed out]; nested: SocketTimeoutException[Read timed out];
I've had that issue previously with 5.x as well, but bumping the read timeout up to 30 minutes had resolved that.
I've tried bumping it all the way up to 120 minutes now, but the only change is that the timeout now happens after 2 hours instead of 30 minutes.
The indices being backed up are smaller now than they used to be (~ 8GB now vs ~ 20GB previously) so I should not be hitting any GCS limits now that I wasn't hitting before, unless the plugin is doing very different things now compared to on the previous ES version.
I've basically already tried everything listed in Snapshot to GCS repository times out so I'm at a loss as to what to try next.