Hi All,
Here is my setup information
ES version: 2.3.3
Spark: 2.0.0
ES-Hadoop jar: elasticsearch-hadoop-2.3.3.jar
ES-Spark Jar: elasticsearch-spark_2.11-2.3.3.jar
I am trying to read data from ES index and trying to perform join using Spark using and again trying to insert in Elasticsearch index.Reading data from ES is fine and join is also fine but when i am trying to insert response (basically something like this "df.write.format("org.elasticsearch.spark.sql").option("es.resource", "combinejoin/join").save()" )
it is throwing error:
Here is my full Stacktrace:
py4j.protocol.Py4JJavaError: An error occurred while calling o34.save.
: java.lang.AbstractMethodError: org.elasticsearch.spark.sql.DefaultSource.createRelation(Lorg/apache/spark/sql/SQLContext;Lorg/apache/spark/sql/SaveMode;Lscala/collection/immutable/Map;Lorg/apache/spark/sql/Dataset;)Lorg/apache/spark/sql/sources/BaseRelation;
at org.apache.spark.sql.execution.datasources.DataSource.write(DataSource.scala:429)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:211)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:237)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:280)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:128)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:211)
at java.lang.Thread.run(Thread.java:745)