Hide parameters in Spark History server


We are trying to connect Spark 3.0.1 with Elasticsearch using the Elastic-hadoop library (Maven Repository: org.elasticsearch » elasticsearch-spark-30) and everything works fine.
However, we are using Spark History server to track jobs. Having a look at the SQL upper tab and seeing the explain details it shows the following:

As you can see, it is showing the user and password which is not very safe :neutral_face:

We know it is possible to use a keystore: Security | Elasticsearch for Apache Hadoop [7.12] | Elastic
But we are retrieving at runtime those credentials. So, we cannot generate the keystore before running the job.

Is it possible to hide this information?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.