What version of elasticsearch-hadoop should I be using with spark 4.0.1

Elastic Stack Elasticsearch
,
1

Using Scala version 2.13.16 (OpenJDK 64-Bit Server VM, Java 17.0.16)

I'm getting this error when writing to an ES 8.19.3 cluster: java.lang.NoSuchMethodError: 'org.apache.spark.sql.SQLContext org.apache.spark.sql.Dataset.sqlContext()'

I was using org.elasticsearch:elasticsearch-spark-30_2.13:8.17.0 before on spark 3.4 with no issues.

I've tried the latest versions of elasticsearch-hadoop for 8.x.x and 9.x.x all giving the above error.

Thank you

2

Spark 4.x is not currently supported unfortunately.