I've been trying to create a snapshot repository in an hdfs install configured with kerberos but cannot get it working. Has anybody gotten this working?
Error (for the commands without uri specified):
{"error":{"root_cause":[{"type":"repository_verification_exception","reason":"[esbackup] path is not accessible on master node"}],"type":"repository_verification_exception","reason":"[esbackup] path is not accessible on master node","caused_by":{"type":"i_o_exception","reason":"Mkdirs failed to create file:/es_snap/tests-IcnM9eSiTLuldgMQrDVXuA (exists=false, cwd=file:/usr/share/elasticsearch)"}},"status":500}
For context:
- Elasticsearch process is running a the correct user that has hdfs access
- ES_JAVA_OPTS has been populated with specifics such as krb5.conf location and pointing to correct KDC.
- As the user that's running elasticsearch, I can create files and folder under the repository directory I'm assigning.
- I can kinit just fine with the keytab provided and can verify all hosts are able to + are defined.
Different combinations tried so far:
{"type":"hdfs","settings":{"path":"/es_snap/","compress":"true","user":"username","user_keytab":"/keytabs/username.keytab"}}
----
{"type":"hdfs","settings":{"path":"/es_snap/","compress":"true","user_principal":"username","user_keytab":"/keytabs/username.keytab"}}
----
{"type":"hdfs","settings":{"path":"/es_snap/","compress":"true"}}
----
{"type":"hdfs","settings":{"uri":"hdfs://hostname.com:8020/","path":"/es_snap/","compress":"true"}}
----
{"type":"hdfs","settings":{"uri":"hdfs://hostname.com:8020/","path":"/es_snap/","compress":"true","user_principal":"username","user_keytab":"/keytabs/username.keytab"}}
Also, I've noticed that if you specify a uri it switches to simple auth vs kerberos?