Elasticsearch-hadoop Spark 4 compatible release timelines

Elastic Stack Elasticsearch
1

Hi,

I'd like to know when a Spark 4 compatible release of GitHub - elastic/elasticsearch-hadoop: Elasticsearch real-time search and analytics natively integrated with Hadoop · GitHub is expected? I believe this might be 9.6?

Thanks in advance.

Nathan

2

Hi can i know the importance to it

3

Hi Etiosa,

We're in the process of upgrading from Spark 3 to Spark 4, and this is one of the few libraries we use which isn't yet publishing Spark 4 versions. If there's a version being released soon (it looks like work has very much started landing into main) then I won't look at replacing it in our stack; if there isn't then I'll have to look at options.

Nathan

4

Yes, support for Spark 3.5, 4.0, 4.1, and 4.2 will be in 9.6.0.

5

Super, and is there a target release date for that?

6

Unfortunately, Elastic does not publish target release dates. But you can probably get a good idea of when it will be based on the cadence of past minor releases.

7

Fair enough, looks like it might be a little while.

Do you publish snapshot/nightlies to anywhere publically accessible?

8

I don't know whether we publish snapshot artifacts anywhere. If you're comfortable building from source, you can clone the repository at GitHub - elastic/elasticsearch-hadoop: Elasticsearch real-time search and analytics natively integrated with Hadoop · GitHub, and then build the spark jar you're interested in with something like ./gradlew :elasticsearch-spark-41:assemble (that 41 is for spark 4.1 support -- change it if you're using a different version of spark). Then you'll find the artifact at ./spark/sql-41/build/libs/elasticsearch-spark-41_2.13-9.6.0-SNAPSHOT.jar.

9

hi everyone, @Keith_Massey thanks for latest advices.

Building from source is OK to start migration to Spark 4.1 locally, but more restrictive for our production context. As of today, did elastic team have any projection on a potential target release date for elasticsearch-spark-41 with version 9.6.0 ?

Hakan

10

We don't publish expected release dates. But based on the regular frequency of our minor releases, you could probably get a pretty good idea of when to expect 9.6.0.

11

@Nathan_Grand cc @Keith_Massey @hakan_gunes

Here are the Daily Release Artifacts, including the Hadoop Plugin

Click on this to get to the actual page and downloads

Screenshot 2026-11-02 at 3.18.31 PM
Screenshot 2026-11-02 at 3.18.31 PM1764×392 48.7 KB

Example latest 9.6.0-SNAPSHOT (as my posting)