# Databricks Spark SQL and Elasticsearch

**URL:** <https://discuss.elastic.co/t/databricks-spark-sql-and-elasticsearch/328028>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [March 19, 2023, 6:37pm UTC](https://discuss.elastic.co/t/databricks-spark-sql-and-elasticsearch/328028 "2023-03-19T18:37:47Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mruthyu](https://avatars.discourse-cdn.com/v4/letter/m/bb73d2/32.png) [@mruthyu](https://discuss.elastic.co/u/mruthyu)\
**Post date:** [March 19, 2023, 6:37pm UTC](https://discuss.elastic.co/t/databricks-spark-sql-and-elasticsearch/328028/1 "2023-03-19T18:37:47Z")

</div>

Is there any documentation related to Databricks Spark/Spark SQL integration with elasticsearch?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 19, 2023, 8:14pm UTC](https://discuss.elastic.co/t/databricks-spark-sql-and-elasticsearch/328028/2 "2023-03-19T20:14:32Z")

</div>

This is not something our documentation covers, no. You might need to ask the databricks team.

---

<div class="post-metadata">

**Author:** ![mruthyu](https://avatars.discourse-cdn.com/v4/letter/m/bb73d2/32.png) [@mruthyu](https://discuss.elastic.co/u/mruthyu)\
**Post date:** [March 20, 2023, 7:03am UTC](https://discuss.elastic.co/t/databricks-spark-sql-and-elasticsearch/328028/3 "2023-03-20T07:03:02Z")

</div>

I could see the documentation related to integration with Hadoop - [Elasticsearch for Hadoop | Elastic](https://www.elastic.co/what-is/elasticsearch-hadoop)

So was checking if there is similar documentation for Databricks/Spark integration as well.

Sure. Thanks.

---

<div class="post-metadata">

**Author:** ![Keith\_Massey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/keith_massey/32/83666_2.png) [@Keith\_Massey](https://discuss.elastic.co/u/Keith_Massey)\
**Post date:** [March 20, 2023, 10:49pm UTC](https://discuss.elastic.co/t/databricks-spark-sql-and-elasticsearch/328028/4 "2023-03-20T22:49:16Z")

</div>

I assume you have seen [Apache Spark support | Elasticsearch for Apache Hadoop [8.6] | Elastic](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/spark.html)? Es-spark is just a library. You just need to figure out how to get it into the classpath of whatever you are using to interact with spark (whether that's spark-shell, your own application, or a notebook). For example to try it out in spark shell (regardless of whether it's Cloudera or Databricks or plain Apache) you can just do something like this:

```auto
/home/keith/spark-3.2.1-bin-hadoop3.2/bin/spark-shell --master yarn --deploy-mode client --jars /home/keith/elasticsearch-spark-30_2.12-8.6.0.jar

```

Configuration for talking with your Elasticsearch cluster is then done in your spark code (see that first link above). If you can describe how you're trying to use it we might be able to give you more help. But as Mark said, we don't document all of the different Spark distributions or ways to use Spark.

---

<div class="post-metadata">

**Author:** ![mruthyu](https://avatars.discourse-cdn.com/v4/letter/m/bb73d2/32.png) [@mruthyu](https://discuss.elastic.co/u/mruthyu)\
**Post date:** [March 22, 2023, 6:19am UTC](https://discuss.elastic.co/t/databricks-spark-sql-and-elasticsearch/328028/5 "2023-03-22T06:19:45Z")

</div>

Thanks Keith. This helps.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 19, 2023, 6:19am UTC](https://discuss.elastic.co/t/databricks-spark-sql-and-elasticsearch/328028/6 "2023-04-19T06:19:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.