ES+Hadoop+Spark 6.6 securable keystore

Elastic Stack Elasticsearch
1

Hello all,

Following the instructions on https://www.elastic.co/guide/en/elasticsearch/hadoop/6.6/security.html#keystore, we have successfully created a keystore.

So far we added to it the key="es.net.http.auth.pass" with a value corresponding to the password.

Now we were instancing our Spark configuration like this:

val conf = new SparkConf().setAppName("ReadFromES")
.set("es.nodes", "xxx.xxx.xxx.xxx")
.set("es.port", "9200")
.set("es.net.ssl", "true")
.set("es.net.ssl.truststore.location", "file:///etc/spark/conf/truststore.jks")
.set("es.net.ssl.truststore.pass", "truststore_password")
.set("es.keystore.location", "file:///etc/spark/conf/esh.keystore")
.set("es.nodes.wan.only", "true")
.set("es.net.http.auth.user", "user_name")

Running it like this, it does find both truststore and keystore (we deployed them on master and worker nodes).

Our question now is: how can we recover the password stored on the keystore being passed, so we can set the configuration key "es.net.http.auth.pass" with the value coming from the keystore?
Is there any naming convention? Do we set that item in a different way?

Thank you.
Best regards,
Miguel

2

Hello guys!

So, is this community dead or doesn't anyone use keystores? :slight_smile:
Or maybe my question is just silly...

Best regards,
Miguel

3

@Miguel_Oliveira sorry your posts haven't received much attention yet. Rest assured that your question is certainly not silly.

Our question now is: how can we recover the password stored on the keystore being passed, so we can set the configuration key "es.net.http.auth.pass" with the value coming from the keystore?
Is there any naming convention? Do we set that item in a different way?

When using the ES-Hadoop secure settings feature (the es.keystore.location setting), the connector will read settings from the keystore as if those settings were provided through the regular configuration for the job.

In this case, you are setting your es.net.http.auth.pass value in the keystore so that it isn't in plain text in the job configuration. When the job starts though, it will attempt to read all securable settings from the keystore first, and if it does not find them in there, it will fall back to checking the regular configuration (for the sake of backwards compatibility). Do note that the only settings that will be read from the keystore in this manner are the following ones:

  • es.net.http.auth.pass
  • es.net.ssl.keystore.pass
  • es.net.ssl.truststore.pass
  • es.net.proxy.http.pass
  • es.net.proxy.https.pass
  • es.net.proxy.socks.pass

Are you currently running into any problems with accessing the cluster?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.