# Connecting elastic search through pyspark

**URL:** <https://discuss.elastic.co/t/connecting-elastic-search-through-pyspark/283170>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [September 2, 2021, 1:22pm UTC](https://discuss.elastic.co/t/connecting-elastic-search-through-pyspark/283170 "2021-09-02T13:22:51Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Destructord007](https://avatars.discourse-cdn.com/v4/letter/d/9de053/32.png) [@Destructord007](https://discuss.elastic.co/u/Destructord007)\
**Post date:** [September 2, 2021, 1:22pm UTC](https://discuss.elastic.co/t/connecting-elastic-search-through-pyspark/283170/1 "2021-09-02T13:22:51Z")

</div>

```auto

def main():
    es_read_conf = {
        "es.nodes": "ES_LINK",
        "es.port": "9200",
        "es.net.http.auth.user": 'ES_USERNAME',
        "es.net.http.auth.pass":'ES_PASSWORD',
    }
    print("ES_CONF_START")
    es_rdd = sc.newAPIHadoopRDD(
        inputFormatClass="org.elasticsearch.hadoop.mr.EsInputFormat",
        keyClass="org.apache.hadoop.io.NullWritable",
        valueClass="org.elasticsearch.hadoop.mr.LinkedMapWritable",
        conf=es_read_conf)
    print("ES_CONF_DONE")

if __name__ == ' __main__':

    conf = SparkConf().setAppName("ESTest")
    print('conf done')
    sc = SparkContext(conf=conf)
    print('sc done')
    sqlContext = SQLContext(sc)
    print('sql done')
    main()

```

java.lang.ClassNotFoundException: org.elasticsearch.hadoop.mr.LinkedMapWritable  
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)  
at java.lang.ClassLoader.loadClass(ClassLoader.java:419)  
at java.lang.ClassLoader.loadClass(ClassLoader.java:352)  
at java.lang.Class.forName0(Native Method)  
at java.lang.Class.forName(Class.java:348)  
at org.apache.spark.util.Utils$.classForName(Utils.scala:237)  
at org.apache.spark.api.python.PythonRDD$.newAPIHadoopRDDFromClassNames(PythonRDD.scala:313)  
I have been trying to connect to elastic search through python facing this issue?can anyone please help?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 30, 2021, 1:23pm UTC](https://discuss.elastic.co/t/connecting-elastic-search-through-pyspark/283170/2 "2021-09-30T13:23:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

---

<div class="post-metadata">

**Author:** ![Keith\_Massey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/keith_massey/32/83666_2.png) [@Keith\_Massey](https://discuss.elastic.co/u/Keith_Massey)\
**Post date:** [December 7, 2021, 6:56pm UTC](https://discuss.elastic.co/t/connecting-elastic-search-through-pyspark/283170/3 "2021-12-07T18:56:32Z")

</div>

Hi @Destructord007. How did you start pyspark. It looks like the es-spark library was not available in your pyspark session. Make sure when you start pyspark that you pass it the location of your es-spark library, something like `--jars /tmp/elasticsearch-spark-30_2.12-7.15.0.jar`