# Problem between Spark and Elasticsearch

**URL:** <https://discuss.elastic.co/t/problem-between-spark-and-elasticsearch/51942>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [June 6, 2016, 12:28pm UTC](https://discuss.elastic.co/t/problem-between-spark-and-elasticsearch/51942 "2016-06-06T12:28:09Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vincent\_Vost](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vincent_vost/32/9946_2.png) [@Vincent\_Vost](https://discuss.elastic.co/u/Vincent_Vost)\
**Post date:** [June 6, 2016, 12:28pm UTC](https://discuss.elastic.co/t/problem-between-spark-and-elasticsearch/51942/1 "2016-06-06T12:28:10Z")

</div>

Hi 🙂

I would like to use Spark with Elasticsearch, but I have some trouble for put everything together

I'm running 2 vm.  
One with elasticsearch and the second with spark.

> import org.apache.spark.SparkConf;  
> import org.apache.spark.api.java.JavaPairRDD;  
> import org.apache.spark.api.java.JavaSparkContext;  
> import org.elasticsearch.spark.rdd.api.java.JavaEsSpark;

> import java.util.Map;

> public class MySparkElas {

> ```
> public static void main(String[] args) {
> 
> ```

> ```
> SparkConf conf = new SparkConf().setAppName("MySparkElas").setMaster("local");
> conf.set("es.index.auto.create", "true");
> conf.set("es.nodes", "192.168.1.14");
> conf.set("es.port", "9200");
> JavaSparkContext sc = new JavaSparkContext(conf);
> 
> ```

> ```
> JavaPairRDD<String, Map<String, Object>> esRDD = JavaEsSpark.esRDD(sc,"test/);
> 
> ```

> ```
> System.out.print("Number of elements : "+esRDD.count());
> }
> 
> ```
> 
> }

And I get this error :

> 16/06/06 12:14:03 INFO BlockManagerMasterEndpoint: Registering block manager localhost:37102 with 517.4 MB RAM, BlockManagerId(driver, localhost, 37102)  
> 16/06/06 12:14:03 INFO BlockManagerMaster: Registered BlockManager  
> Exception in thread "main" java.lang.NoClassDefFoundError: org/elasticsearch/spark/rdd/api/java/JavaEsSpark  
> at com.seigneurin.spark.MySparkElas.main(MySparkElas.java:25)  
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)  
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)  
> at java.lang.reflect.Method.invoke(Method.java:498)  
> at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)  
> at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)  
> at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)  
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)  
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)  
> Caused by: java.lang.ClassNotFoundException: org.elasticsearch.spark.rdd.api.java.JavaEsSpark  
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)  
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)  
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)  
> ... 10 more

Do you have an idea why it didn't work ?  
Thank you 🙂

---

<div class="post-metadata">

**Author:** ![Vincent\_Vost](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vincent_vost/32/9946_2.png) [@Vincent\_Vost](https://discuss.elastic.co/u/Vincent_Vost)\
**Post date:** [June 6, 2016, 12:56pm UTC](https://discuss.elastic.co/t/problem-between-spark-and-elasticsearch/51942/2 "2016-06-06T12:56:57Z")

</div>

Ok! I found the problem.

I didn't take the dependencies with the JAR. Now I have an other error, but this one is okay 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:24pm UTC](https://discuss.elastic.co/t/problem-between-spark-and-elasticsearch/51942/3 "2017-07-06T13:24:27Z")

</div>