# Java security apparent bug with repository-gcs **URL:** \ **Category:** Elasticsearch\ **Created:** [May 8, 2019, 9:30am UTC](https://discuss.elastic.co/t/java-security-apparent-bug-with-repository-gcs/180144 "2019-05-08T09:30:26Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![MichaelSGrant](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michaelsgrant/32/45802_2.png) [@MichaelSGrant](https://discuss.elastic.co/u/MichaelSGrant)\ **Post date:** [May 8, 2019, 9:30am UTC](https://discuss.elastic.co/t/java-security-apparent-bug-with-repository-gcs/180144/1 "2019-05-08T09:30:26Z")
I have a Elasticsearch cluster which I have just migrated from 5.6.4 to 6.5.1. (This version was selected because it is the highest version currently supported by the `elastic4s` library used by our code.) Unfortunately, this has caused a problem with snapshotting; when I try it I get a `java.lang.SecurityException`: `access denied ("java.lang.RuntimePermission" "accessDeclaredMembers")`. (It all worked fine before the upgrade.) We're running Elasticsearch in a Docker container (on Linux) based on `elasticsearch:6.5.1`, but I have also seen the problem when running Elasticsearch standalone (on MacOS); it also occurs whether running with multiple nodes (in our staging environment) or a single node (on my Macbook). It also occurs in later 6.x versions. To reproduce the problem it is not necessary to do any more than run Elasticsearch with the `repository-gcs` plugin installed, then `PUT` `{ "type":"gcs", "settings": {"bucket":"elasticsearch-backup-store-[REDACTED]-staging"}}` to `/_snapshot/elasticsearch_backup_repository`. The problem occurs despite the `grant { [...] permission java.lang.RuntimePermission "accessDeclaredMembers"; [...] };` in `/usr/share/elasticsearch/plugins/repository-gcs/plugin-security.policy`; it even occurs if I try to add the grant globally in `/opt/jdk-11.0.1/conf/security/java.policy` or `/opt/jdk-11.0.1/lib/security/default.policy`. I tried running ES with `-Djava.security.debug=access,failure`, and observed a curious thing: Whilst other security grant denials were reported with information about the domain that caused the security breach, this one was not, which suggests to me that possibly `repository-gcs` is not running with the same security setup as the main Elasticsearch executable. Here's the full stacktrace of the problem (edited down slightly to get under the character limit): ```auto [INFO][o.e.r.RepositoriesService] [kPIR_P5] put repository [elasticsearch_backup_repository] [WARN][o.e.r.g.GoogleCloudStorageService] [kPIR_P5] "Application Default Credentials" are not supported out of the box. Additional file system permissions have to be granted to the plugin. [WARN][r.suppressed] [kPIR_P5] path: /_snapshot/elasticsearch_backup_repository, params: {repository=elasticsearch_backup_repository} org.elasticsearch.repositories.RepositoryException: [elasticsearch_backup_repository] cannot create blob store at org.elasticsearch.repositories.blobstore.BlobStoreRepository.blobStore(BlobStoreRepository.java:336) ~[elasticsearch-6.5.1.jar:6.5.1] at o.e.repositories.blobstore.BlobStoreRepository.startVerification(BlobStoreRepository.java:635) ~[elasticsearch-6.5.1.jar:6.5.1] at o.e.repositories.RepositoriesService.lambda$verifyRepository$2(RepositoriesService.java:218) [elasticsearch-6.5.1.jar:6.5.1] at o.e.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:624) [elasticsearch-6.5.1.jar:6.5.1] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.lang.Thread.run(Thread.java:834) Caused by: org.elasticsearch.common.blobstore.BlobStoreException: Unable to check if bucket [elasticsearch-backup-store-redimapl-staging] exists at o.e.repositories.gcs.GoogleCloudStorageBlobStore.doesBucketExist(GoogleCloudStorageBlobStore.java:111) at o.e.repositories.gcs.GoogleCloudStorageBlobStore.(GoogleCloudStorageBlobStore.java:77) at o.e.repositories.gcs.GoogleCloudStorageRepository.createBlobStore(GoogleCloudStorageRepository.java:135) at o.e.repositories.gcs.GoogleCloudStorageRepository.createBlobStore(GoogleCloudStorageRepository.java:46) at o.e.repositories.blobstore.BlobStoreRepository.blobStore(BlobStoreRepository.java:332) ~[elasticsearch-6.5.1.jar:6.5.1] ... 6 more Caused by: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessDeclaredMembers") at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) at java.security.AccessController.checkPermission(AccessController.java:895) at java.lang.SecurityManager.checkPermission(SecurityManager.java:322) at java.lang.Class.checkMemberAccess(Class.java:2848) at java.lang.Class.getDeclaredFields(Class.java:2247) at com.google.api.client.util.ClassInfo.(ClassInfo.java:171) at c.g.a.c.util.ClassInfo.of(ClassInfo.java:90) at c.g.a.c.util.ClassInfo.(ClassInfo.java:193) at c.g.a.c.util.ClassInfo.of(ClassInfo.java:90) at c.g.a.c.util.ClassInfo.(ClassInfo.java:193) at c.g.a.c.util.ClassInfo.of(ClassInfo.java:90) at c.g.a.c.util.GenericData.(GenericData.java:79) at c.g.a.c.util.GenericData.(GenericData.java:61) at c.g.a.c.http.GenericUrl.(GenericUrl.java:157) at c.g.a.c.http.GenericUrl.(GenericUrl.java:142) at c.g.a.c.http.GenericUrl.(GenericUrl.java:114) at com.google.cloud.ServiceOptions.getAppEngineProjectIdFromMetadataServer(ServiceOptions.java:452) at c.g.c.ServiceOptions.getAppEngineProjectId(ServiceOptions.java:431) at c.g.c.ServiceOptions.getDefaultProjectId(ServiceOptions.java:339) at c.g.c.ServiceOptions.getDefaultProject(ServiceOptions.java:317) at c.g.c.ServiceOptions.(ServiceOptions.java:270) at c.g.c.storage.StorageOptions.(StorageOptions.java:83) at c.g.c.storage.StorageOptions.(StorageOptions.java:31) at c.g.c.storage.StorageOptions$Builder.build(StorageOptions.java:78) at org.elasticsearch.repositories.gcs.GoogleCloudStorageService.createClient(GoogleCloudStorageService.java:211) at o.e.repositories.gcs.GoogleCloudStorageService.lambda$refreshAndClearCache$0(GoogleCloudStorageService.java:89) at o.e.common.util.LazyInitializable.maybeCompute(LazyInitializable.java:103) ~[elasticsearch-6.5.1.jar:6.5.1] at o.e.common.util.LazyInitializable.getOrCompute(LazyInitializable.java:81) ~[elasticsearch-6.5.1.jar:6.5.1] at o.e.repositories.gcs.GoogleCloudStorageService.client(GoogleCloudStorageService.java:114) at o.e.repositories.gcs.GoogleCloudStorageBlobStore.client(GoogleCloudStorageBlobStore.java:83) at o.e.repositories.gcs.GoogleCloudStorageBlobStore.lambda$doesBucketExist$0(GoogleCloudStorageBlobStore.java:108) at java.security.AccessController.doPrivileged(Native Method) at o.e.repositories.gcs.SocketAccess.doPrivilegedIOException(SocketAccess.java:44) at o.e.repositories.gcs.GoogleCloudStorageBlobStore.doesBucketExist(GoogleCloudStorageBlobStore.java:108) at o.e.repositories.gcs.GoogleCloudStorageBlobStore.(GoogleCloudStorageBlobStore.java:77) at o.e.repositories.gcs.GoogleCloudStorageRepository.createBlobStore(GoogleCloudStorageRepository.java:135) at o.e.repositories.gcs.GoogleCloudStorageRepository.createBlobStore(GoogleCloudStorageRepository.java:46) at o.e.repositories.blobstore.BlobStoreRepository.blobStore(BlobStoreRepository.java:332) ~[elasticsearch-6.5.1.jar:6.5.1] ... 6 more ``` Any help would be appreciated. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 8, 2019, 9:32am UTC](https://discuss.elastic.co/t/java-security-apparent-bug-with-repository-gcs/180144/2 "2019-05-08T09:32:51Z")
---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 8, 2019, 9:33am UTC](https://discuss.elastic.co/t/java-security-apparent-bug-with-repository-gcs/180144/3 "2019-05-08T09:33:37Z")
I've made this unlisted, aka private, while I ask our security team to look at this. ---
**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\ **Post date:** [May 8, 2019, 11:08am UTC](https://discuss.elastic.co/t/java-security-apparent-bug-with-repository-gcs/180144/4 "2019-05-08T11:08:08Z")
Hi there, This fails as you haven't provided any client authentication credentials for the plugin: ```auto [WARN][o.e.r.g.GoogleCloudStorageService] [kPIR_P5] "Application Default Credentials" are not supported out of the box. Additional file system permissions have to be granted to the plugin. ``` please see [our docs](https://www.elastic.co/guide/en/elasticsearch/plugins/6.5/repository-gcs-usage.html#repository-gcs-service-authentication) where we explain why we can't work with application default credentials and what are your options. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 8, 2019, 11:11am UTC](https://discuss.elastic.co/t/java-security-apparent-bug-with-repository-gcs/180144/5 "2019-05-08T11:11:59Z")
---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 5, 2019, 11:12am UTC](https://discuss.elastic.co/t/java-security-apparent-bug-with-repository-gcs/180144/6 "2019-06-05T11:12:02Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.