This is the line I use:
.write.options(**es_write_conf).mode("append").format("org.elasticsearch.spark.sql").save(esPath)
When I run it in a shell launching it with pyspark -jars /usr/lib/spark/jars/elasticsearch-hadoop-5.6.9.jar
it runs perfectly. But when I tried to run it through spark-submit -jars /usr/lib/spark/jars/elasticsearch-hadoop-5.6.9.jar
I get a ClassNotFoundException.
What would I need to do to figure out why it doesn't work on spark-submit?