I could Not make it work with the PKCS12 type in table properties like this
"es.net.ssl.keystore.type"="PKCS12",
"es.net.ssl.truststore.location"="file:/path_to_cert_on_local/http.p12",
How to make it work in es-hadoop v.7.8.0 with es v.7.8.0 ?
I didn't find any successful solution on the forum.
I made the following configuration work successfully with the JKS type:
Imported the http.crt into truststore.jks with the command
keytool -import -trustcacerts -alias tls -file http.crt -keystore truststore.jks
and the truststore.jks has been generated
hive> create external table if not exists db_name.tbl_name (
id string,
title string,
year string)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES(
"es.nodes"="xx.xxx.xx.xxx",
"es.port"="xxxx",
"es.resource.read"="books/book",
"es.read.operation"="index",
"es.index.read.missing.as.empty"="true",
"es.nodes.discovery"="true",
"es.read.metadata"="true",
"es.mapping.names"="id:_metadata._id, title:title, year:year",
"es.net.ssl"="true",
"es.net.ssl.cert.allow.self.signed"="true",
"es.net.ssl.keystore.type"="jks",
"es.net.ssl.truststore.location"="file:/xxxxx_domain/tmp/truststore.jks",
"es.net.http.auth.user"="estester",
"es.net.http.auth.pass"="estester_pwd");
This configuration works well, I was able to run select from table successfully, but I'd like to take off the user/password in plain text from table properties and incorporate them into keystore file and reference this file in table properties
Importing user/password settings into keystore file - Not working
To import "es.net.http.auth.user"="estester" and "es.net.http.auth.pass"="estester_pwd" settings into keystore file I followed up
To create keystore file I ran the command as guided in the documentation
java -classpath elasticsearch-hadoop-7.8.0.jar org.elasticsearch.hadoop.cli.keytool create
and esh.keystore file has been created; than I ran
java -classpath elasticsearch-hadoop-7.8.0.jar org.elasticsearch.hadoop.cli.keytool add es.net.http.auth.user
enter value for es.net.http.auth.user: [entered estester]
java -classpath elasticsearch-hadoop-7.8.0.jar org.elasticsearch.hadoop.cli.keytool add es.net.http.auth.pass
enter value for es.net.http.auth.pass: [entered estester_pwd]
and in table properties I used
"es.net.ssl.keystore.location"="file:/xxxxx_domain/tmp/esh.keystore"
instead of "es.net.http.auth.user"="estester" and "es.net.http.auth.pass"="estester_pwd"
I added file
hive> ADD FILE /path_to_file/esh.keystore;
Recreated table, did select from table and log shows the Exception
.....
java.io.IOException:org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version- this typically happens if network/elasticsearch cluster is not accessible or when targetting a WAN/Cloud instance without proper setting in 'es.nodes.wan.only'
.....
Caused by: org.elasticsearch.hadoop.EsHadoopIllegalState exception: Cannot initialize SSL - Get Key failed: AES SecretKeyFactory not available
.....
Caused by: java.security.UnrecoverableKeyException: Get Key failed: AES SecretKeyFactory not available
.....
Caused by: java.security.NoSuchAlgorithmException: AES SecretKeyFactory not available
Q:How to create keystore file properly and import user/password into it ?
Q2: Which value should be assigned to "es.net.ssl.keystore.pass" and where it comes from if it is needed at all ?