Elasticsearch cannot find a non-existing file

Elastic Stack Elasticsearch
1

Hello,

after enabling and configuring ES to use s3 repository plugin when we restart each node we got a following error right after restarting nodes and only. When I enter the directory the file is missing. Els process is started with the same user dbolog.

[2020-11-17T08:47:32,263][WARN ][c.a.a.p.i.BasicProfileConfigFileLoader] [logstashatp3] Unable to load config file null
java.security.AccessControlException: access denied ("java.io.FilePermission" "/home/dbolog/.aws/config" "read")
        at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]
        at java.security.AccessController.checkPermission(AccessController.java:895) ~[?:?]
        at java.lang.SecurityManager.checkPermission(SecurityManager.java:322) ~[?:?]
        at java.lang.SecurityManager.checkRead(SecurityManager.java:661) ~[?:?]
        at java.io.File.exists(File.java:815) ~[?:?]
        at com.amazonaws.profile.path.config.SharedConfigDefaultLocationProvider.getLocation(SharedConfigDefaultLocationProvider.java:36) ~[aws-java-sdk-core-1.11.749.jar:?]
        at com.amazonaws.profile.path.AwsProfileFileLocationProviderChain.getLocation(AwsProfileFileLocationProviderChain.java:41) ~[aws-java-sdk-core-1.11.749.jar:?]
        at com.amazonaws.auth.profile.internal.BasicProfileConfigFileLoader.getProfilesConfigFile(BasicProfileConfigFileLoader.java:69) [aws-java-sdk-core-1.11.749.jar:?]
        at com.amazonaws.auth.profile.internal.BasicProfileConfigFileLoader.getProfile(BasicProfileConfigFileLoader.java:55) [aws-java-sdk-core-1.11.749.jar:?]
        at com.amazonaws.retry.internal.RetryModeResolver.profile(RetryModeResolver.java:92) [aws-java-sdk-core-1.11.749.jar:?]
        at com.amazonaws.retry.internal.RetryModeResolver.resolveRetryMode(RetryModeResolver.java:83) [aws-java-sdk-core-1.11.749.jar:?]
        at com.amazonaws.retry.internal.RetryModeResolver.<init>(RetryModeResolver.java:46) [aws-java-sdk-core-1.11.749.jar:?]
        at com.amazonaws.retry.RetryPolicy.<clinit>(RetryPolicy.java:35) [aws-java-sdk-core-1.11.749.jar:?]
        at com.amazonaws.retry.PredefinedRetryPolicies.<clinit>(PredefinedRetryPolicies.java:30) [aws-java-sdk-core-1.11.749.jar:?]
        at com.amazonaws.ClientConfiguration.<clinit>(ClientConfiguration.java:89) [aws-java-sdk-core-1.11.749.jar:?]
        at java.lang.Class.forName0(Native Method) [?:?]
        at java.lang.Class.forName(Class.java:315) [?:?]
        at org.elasticsearch.repositories.s3.S3RepositoryPlugin.lambda$static$0(S3RepositoryPlugin.java:58) [repository-s3-7.9.2.jar:7.9.2]
        at java.security.AccessController.doPrivileged(Native Method) [?:?]
        at org.elasticsearch.repositories.s3.S3RepositoryPlugin.<clinit>(S3RepositoryPlugin.java:52) [repository-s3-7.9.2.jar:7.9.2]
        at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?]
        at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:693) [elasticsearch-7.9.2.jar:7.9.2]
        at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:642) [elasticsearch-7.9.2.jar:7.9.2]
        at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:473) [elasticsearch-7.9.2.jar:7.9.2]
        at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:165) [elasticsearch-7.9.2.jar:7.9.2]
        at org.elasticsearch.node.Node.<init>(Node.java:328) [elasticsearch-7.9.2.jar:7.9.2]
        at org.elasticsearch.node.Node.<init>(Node.java:277) [elasticsearch-7.9.2.jar:7.9.2]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:227) [elasticsearch-7.9.2.jar:7.9.2]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227) [elasticsearch-7.9.2.jar:7.9.2]
2

Did you define the S3 credentials in your configuration?

I suspect that because no configuration has been found, the AWS SDK is trying to access a default folder.

Could you share your elasticsearch.yml configuration?

See also

3

This is in our elasticsearch.yml:

# -------------------------------- S3 backup --------------------------------
s3.client.default.endpoint: "*****.net"
s3.client.default.protocol: "https"

And in elasticsearch.keystore we add:

s3.client.default.access_key: "***************"
s3.client.default.secret_key: "****************************"
4

When you installed the plugin, did you confirm the security policy changes?

   @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
   @     WARNING: plugin requires additional permissions     @
   @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
   * java.lang.RuntimePermission accessDeclaredMembers
   * java.lang.RuntimePermission getClassLoader
   * java.lang.reflect.ReflectPermission suppressAccessChecks
   * java.net.SocketPermission * connect,resolve
   * java.util.PropertyPermission es.allow_insecure_settings read,write
   See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
   for descriptions of what these permissions allow and the associated risks.
   
   Continue with installation? [y/N]y
5

Hi @vaclav1

this is a known issue and has been fixed in 7.10. It should just be cosmetic in nature and you can ignore the warning. See below fix

1 Like
6

Thank you!!

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.