How can i configure hdfs repository with kerberos in a security hdfs cluster?

Elastic Stack Elasticsearch
1

Here is my settings, but it doesn't work (seems the same with repository-hdfs no work with Kerberos authenticate hdfs · Issue #921 · elastic/elasticsearch-hadoop · GitHub).

{
"type": "hdfs",
"settings": {
"path": "/user/me/es_repo",
"compress": "true",
"uri": "hdfs://namenode:8020",
"max_snapshot_bytes_per_sec": "500mb",
"max_restore_bytes_per_sec": "500mb",
"conf_location": "/path/to/my/core-site.xml,/path/to/my/hdfs-site.xml",
"user_keytab": "/data/keytab/user.keytab"
}
}

below is the error,

{
"error": {
"root_cause": [
{
"type": "generation_exception",
"reason": "Cannot create HDFS repository for uri [hdfs://100.106.75.147:25000]"
}
],
"type": "repository_exception",
"reason": "[hdfs_repository] failed to create repository",
"caused_by": {
"type": "generation_exception",
"reason": "Cannot create HDFS repository for uri [hdfs://100.106.75.147:25000]",
"caused_by": {
"type": "i_o_exception",
"reason": "SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]",
"caused_by": {
"type": "i_o_exception",
"reason": "SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]"
}
}
}
},
"status": 500
}

2

i'm using elasticsearch-5.1.1.

3

@xy_xin Kerberos authentication for the HDFS Repository Snapshot plugin does not function correctly in 5.x at the moment. There is currently an effort to re-add this functionality so that it may function well with the internal security manager.

4

Thanks for your information @james.baiera.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.