Cannot load class org.elasticsearch.spark.sql.SparkSQLCompatibilityLevel


I am running a Java Spark application. I can run the application from inside the intellij IDE. However, if I try to run it using the application built as a jar I get the following error.

Caused by: org.elasticsearch.hadoop.EsHadoopIllegalStateException: Cannot load class [org.elasticsearch.spark.sql.SparkSQLCompatibilityLevel]

spark-core_2.10 v 1.6.3
elasticsearch-spark-13_2.10 v 5.6.8

The problem seems identical to the one encountered in this thread.

However, no mention is given of how to fix this.

(James Baiera) #2

Since the job succeeds when launched from the IDE but not when built as a jar, I would double check that the ES-Spark jars are indeed available on the classpath for your spark program.

(Thomas Decaux) #3


The cause is the class is loaded dynamically, so this can bug if you optimize the jar (or minimize it via Maven share plugin for instance).

(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.