Where are Keystore and Truststore: How can Spark communicate with ECK by using SSL

Elastic Orchestration Elastic Cloud on Kubernetes (ECK)
1

Hello,

I followed this link to install ECK and now I want to use Spark with ECK by using SSL. I read the following documentation. However for the following configuration I don't know where to find the keystore and truststore files.

  • es.net.ssl.keystore.type
  • es.net.ssl.truststore.location
  • es.net.ssl.truststore.pass

Can someone explain where to find them ?

Thanks

2

Hi,

You have to create the keystore. Assuming you are using the default, self signed, certificates with ECK you can find more information about how to get CA cert here: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-accessing-elastic-services.html#k8s-request-elasticsearch-endpoint

You can then create a keystore, mount it as a secret if Spark is deployed in K8S and set the correct values for Elasticsearch for Hadoop.

1 Like
3

Hi @michael.morello, thanks for your reply !
Can you recommend documentation to create the keystore from CA ? I found ones but it didn't work :frowning: .
Futhermore, do I need to create a truststore as well if i use the self-signed certificate ?

Regards

4

Sorry, I think you should actually add the CA to a truststore, not a keystore

Also note that it will only work if you access Elasticsearch from the k8s cluster since the certificates, by default, only contains some predefined subjects like:

  • cluster-name
  • cluster-name.cluster-namespace
  • cluster-name.cluster-namespace.svc

See here if you want to add your own subjects.

re. the way to create a keystore I think that something like keytool -import -alias ca -file /path/to/ca.crt -keystore cacerts -storepass changethispassword should work

5

Thanks @michael.morello, I achieved to generate the truststore.jks file and it can be read by my spark job. However, I got an error "Invalid keystore format". I will open a new topic about this error.