# Spark ElasticSearch

**URL:** https://discuss.elastic.co/t/spark-elasticsearch/49595
**Category:** Elasticsearch
**Tags:** es-hadoop
**Created:** [May 9, 2016, 10:45pm UTC](https://discuss.elastic.co/t/spark-elasticsearch/49595 "2016-05-09T22:45:36Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![gauravprasad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gauravprasad/32/9677_2.png) [@gauravprasad](https://discuss.elastic.co/u/gauravprasad)
#### Post date: [May 9, 2016, 10:45pm UTC](https://discuss.elastic.co/t/spark-elasticsearch/49595/1 "2016-05-09T22:45:37Z")

</div>

While Fetching the record from ElasticSearch using spark I m getting bellow error.

`Caused by: org.apache.spark.util.TaskCompletionListenerException: ActionRequestValidationException[Validation Failed: 1: no scroll ids specified;] at org.apache.spark.TaskContextImpl.markTaskCompleted(TaskContextImpl.scala:87) at org.apache.spark.scheduler.Task.run(Task.scala:90) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)`  
The code read as Below :

val esConf = Map(ConfigurationOptions.ES\_NODES -\> "n01ssl101.aap.csaa.pri" ,ConfigurationOptions.ES\_PORT-\> "9200",ConfigurationOptions.ES\_HTTP\_TIMEOUT -\> "5m")

sqlContext.read.format("es").options(esConf).load("contract/data").collect

---

<div class="post-metadata">

### Author: ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)
#### Post date: [May 10, 2016, 9:06am UTC](https://discuss.elastic.co/t/spark-elasticsearch/49595/2 "2016-05-10T09:06:53Z")

</div>

Looks like the read cannot be completed for some reason. Can you provide more information about your setup and what library you are using?  
See [this page](https://www.elastic.co/guide/en/elasticsearch/hadoop/master/troubleshooting.html) for more information.

---

<div class="post-metadata">

### Author: ![gauravprasad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gauravprasad/32/9677_2.png) [@gauravprasad](https://discuss.elastic.co/u/gauravprasad)
#### Post date: [May 10, 2016, 6:21pm UTC](https://discuss.elastic.co/t/spark-elasticsearch/49595/3 "2016-05-10T18:21:44Z")

</div>

Hi Costin,

Thanks for the response. ES Jar details is "org.elasticsearch" % "elasticsearch-spark\_2.10" % "2.2.0", .Here is the code line :

`val esConf = Map(ConfigurationOptions.ES_NODES -> "n01ssl101.aap.csaa.pri" ,ConfigurationOptions.ES_PORT-> "9200",ConfigurationOptions.ES_HTTP_TIMEOUT -> "5m") // var esContractRDD = EsSpark.esJsonRDD(sc,"contract/data",esConf) val sqlContext = new SQLContext(sc) val df = sqlContext.read.format("es").options(esConf).load("contract/data")`

And is the sbt file details which talks about the version.

`name := "UpadteMembership"

version := "1.0"

scalaVersion := "2.11.8"

libraryDependencies ++= Seq(  
"org.apache.spark" % "spark-core\_2.10" % "1.5.2",  
"org.apache.hadoop" % "hadoop-common" % "2.7.1" excludeAll ExclusionRule(organization = "javax.servlet"),  
"org.apache.spark" % "spark-sql\_2.10" % "1.5.2",  
"org.apache.spark" % "spark-hive\_2.10" % "1.5.2",  
"org.apache.spark" % "spark-yarn\_2.10" % "1.5.2",  
"org.elasticsearch" % "elasticsearch-spark\_2.10" % "2.2.0",  
"com.databricks" % "spark-csv\_2.10" % "1.3.0",  
"com.databricks" % "spark-xml\_2.10" % "0.3.2",  
"log4j" % "log4j" % "1.2.17",  
"org.apache.spark" % "spark-streaming\_2.10" % "1.5.2"  
)

resolvers ++= Seq(  
"Hortonworks Repository" at "[http://repo.hortonworks.com/content/repositories/releases/](http://repo.hortonworks.com/content/repositories/releases/)",  
"JBoss Repository" at "[http://repository.jboss.org/nexus/content/repositories/releases/](http://repository.jboss.org/nexus/content/repositories/releases/)",  
"Spray Repository" at "[http://repo.spray.cc/](http://repo.spray.cc/)",  
"Cloudera Repository" at "[https://repository.cloudera.com/artifactory/cloudera-repos/](https://repository.cloudera.com/artifactory/cloudera-repos/)",  
"Akka Repository" at "[http://repo.akka.io/releases/](http://repo.akka.io/releases/)",  
"Twitter4J Repository" at "[http://twitter4j.org/maven2/](http://twitter4j.org/maven2/)",  
"Apache HBase" at "[https://repository.apache.org/content/repositories/releases](https://repository.apache.org/content/repositories/releases)",  
"Twitter Maven Repo" at "[http://maven.twttr.com/](http://maven.twttr.com/)",  
"scala-tools" at "[https://oss.sonatype.org/content/groups/scala-tools](https://oss.sonatype.org/content/groups/scala-tools)",  
"Typesafe repository" at "[http://repo.typesafe.com/typesafe/releases/](http://repo.typesafe.com/typesafe/releases/)",  
"Second Typesafe repo" at "[http://repo.typesafe.com/typesafe/maven-releases/](http://repo.typesafe.com/typesafe/maven-releases/)",  
"Mesosphere Public Repository" at "[http://downloads.mesosphere.io/maven](http://downloads.mesosphere.io/maven)",  
Resolver.sonatypeRepo("public")  
)`

---

<div class="post-metadata">

### Author: ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)
#### Post date: [May 18, 2016, 6:46am UTC](https://discuss.elastic.co/t/spark-elasticsearch/49595/4 "2016-05-18T06:46:34Z")

</div>

Looks like you are mixing Scala 2.10 and 2.11. Make sure to use only one.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:24pm UTC](https://discuss.elastic.co/t/spark-elasticsearch/49595/5 "2017-07-06T13:24:42Z")

</div>


