Save Spark Dataframe into ES - Can't handle type exception

[costin (Costin Leau)]

Got to the bottom of it. Applied a fix in master and 2.x - will be included in the upcoming release.

[eliasah]

You rock man, thanks! I don't know if there is a way on how to set this issue as solved nevertheless for other users who might stumble into something like this.

See you next week @Elastic{On} in Paris!

[Jonathan_Spooner]

I believe I'm seeing a similar issue to this thread. Would you be able to comment on Task exception could not be deserialized

[Jonathan_Spooner]

Hi,

I believe my error as described below is related to this error. Would you be able to comment on this issue?

Task exception could not be deserialized

I have a Spark job that runs on my localhost but when run on EMR I'm getting a Warning for WARN ThrowableSerializationWrapper: Task exception could not be deserialized java.lang.ClassNotFoundException: org.elasticsearch.hadoop.EsHadoopException and error 16/03/31 16:32:55 ERROR TaskResultGetter: Could not deserialize TaskEndReason: ClassNotFound with classloader org.apache.spark.util.MutableURLClassLoader@216b55e2 • My Spark EMR version is 1.5.2 • I was using elasticsearch-spark_2.10-2.2.0-…

[costin (Costin Leau)]

@Jonathan_Spooner I've already replied to your post. Let's continue the discussion there and not on this 5 months old thread.