ES-Hadoop compatibility with Spark 2.4.4

Jonas555 · September 21, 2020, 4:08pm

Hello,
We have an EMR v5.28 cluster that comes with Spark 2.4.4. Our understanding is that Elasticsearch-Hadoop is not compatible with Spark 2.4.4 since Spark is compiled with Scala 2.12, and ES-Hadoop requires Scala 2.11. Is that correct? If so, are there any recommendations how we can overcome this pitfall?

Thanks.

system · October 19, 2020, 4:09pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Elasticsearch support for spark 2.4.2 with scala 2.12 Elasticsearch es-hadoop	2	2259	October 4, 2019
SparkSQL to Elasticsearch compatibility problem Elasticsearch es-hadoop	2	1317	April 13, 2017
Any plan to support Spark2.0 in Elasticsearch-Hadoop? Elasticsearch es-hadoop	3	870	January 12, 2017
Elastic Search Spark Connector for Spark 1.6, ES 5.1.1 and Scala 2.10 Elasticsearch es-hadoop	3	2693	March 6, 2017
Spark version, elasticsearch-hadoop version, akka version sync up Elasticsearch	4	646	July 6, 2017

ES-Hadoop compatibility with Spark 2.4.4

Related topics