How to create Keystore and Truststore from CA Certificates

Elastic Orchestration Elastic Cloud on Kubernetes (ECK)
1

Hello folks,

I installed ECK on GKE. I retrieved certificate tls.crt with the following command

kubectl get secret "hulk-es-http-certs-public" -o go-template='{{index .data "tls.crt" | base64decode }}' > tls.crt

I generated a truststore.jks file with the following command:

keytool -import -trustcacerts -alias tls -file tls.crt -keystore truststore.jks

Then I created a secret to transmit to give to my Spark Scala job :

apiVersion: v1
kind: Secret
metadata:
  name: elasticsearch-truststore-secret
  namespace: dev
type: Opaque
data:
  truststore.jks: <<content of truststore.jks in base64>>

But I get the following error:

Caused by: org.elasticsearch.hadoop.EsHadoopIllegalStateException: 
Cannot initialize SSL - Invalid keystore format at ...

What am I doing wrong in creating my truststore.jks file ?

Thanks,

2

Did you create the secret manually by base64 encoding the trust store yourself? Some base64 encoding tools insert extra bytes to pad the data and that could be the issue here. Try running the following command to create the secret instead:

kubectl create secret generic elasticsearch-truststore-secret --from-file=truststore.jks -n dev
1 Like
3

Thanks @charith-elastic, however it still does not work :cry: !
Really I don't know what to try else

4

It's hard to debug your issue because it happens in a different application. I don't think it's a problem with the way ECK generates certificates as I have managed to run a local Spark job using an ECK-managed Elasticsearch cluster and its certificates. In theory, that should work the same way inside a Kubernetes cluster as well. A couple of things you could check are:

  • Ensure that your secret is getting mounted properly into the Spark containers and that the trust store file is present in the location you expect it to be. It's possible that Spark is trying to read the trust store from the wrong path. (https://kubernetes.io/docs/tasks/debug-application-cluster/get-shell-running-container/)
  • Ensure that your trust store secret is not corrupt.
     kubectl get secret elasticsearch-truststore-secret -o=go-template='{{index  .data "truststore.jks" | base64decode }}' | xxd | diff -y <(xxd truststore.jks) -

Good luck!

1 Like
5

@charith-elastic, Thanks for the tips, I achieved to solve my problem :partying_face: !
It was a wrong path to the location of my truststore file.
I set the wrong following configuration in spark

--conf spark.kubernetes.driver.secrets.elasticsearch-truststore-secret=/etc/secrets/trutstore.jks
--conf spark.kubernetes.driver.secrets.elasticsearch-truststore-secret=/etc/secrets/trutstore.jks

Then, it created a folder named trutstore.jks not a file.

So I just changed for:

--conf spark.kubernetes.driver.secrets.elasticsearch-truststore-secret=/etc/secrets/trutstore
--conf spark.kubernetes.driver.secrets.elasticsearch-truststore-secret=/etc/secrets/trutstore

And the right location is /etc/secrets/trutstore/trutstore.jks

Yassir