# Spark task are failing: No search context found for id

**URL:** <https://discuss.elastic.co/t/spark-task-are-failing-no-search-context-found-for-id/163085>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [January 6, 2019, 3:43pm UTC](https://discuss.elastic.co/t/spark-task-are-failing-no-search-context-found-for-id/163085 "2019-01-06T15:43:48Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Satendrakumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/satendrakumar/32/11506_2.png) [@Satendrakumar](https://discuss.elastic.co/u/Satendrakumar)\
**Post date:** [January 6, 2019, 3:43pm UTC](https://discuss.elastic.co/t/spark-task-are-failing-no-search-context-found-for-id/163085/1 "2019-01-06T15:43:49Z")

</div>

Spark tasks are failing because of "No search context found for id".  
I tried a couple of options like  
spark.es.input.max.docs.per.partition 250  
spark.es.scroll.size 100  
spark.es.batch.size.bytes 32mb

But tasks are still failing. we are using:  
Spark version: 2.3.1,  
Elasticsearh Cluster version: 6.5.4([Elastic.co](http://Elastic.co) hosted service 30\*2 GB deployment),  
elasticsearch-spark version: 6.5.4.

```auto
org.elasticsearch.hadoop.rest.EsHadoopInvalidRequest: org.elasticsearch.hadoop.rest.EsHadoopRemoteException: search_context_missing_exception: No search context found for id [4872076]
{"scroll_id":"DXF1ZXJ5QW5kRmV0Y2gBAAAAAABKV4wWRTJHQW9CdEZRTHVqMldEWnQxTUJnQQ=="}
   at org.elasticsearch.hadoop.rest.RestClient.checkResponse(RestClient.java:443)
   at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:400)
   at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:382)
   at org.elasticsearch.hadoop.rest.RestClient.scroll(RestClient.java:458)
   at org.elasticsearch.hadoop.rest.RestRepository.scroll(RestRepository.java:323)
   at org.elasticsearch.hadoop.rest.ScrollQuery.hasNext(ScrollQuery.java:115)
   at org.elasticsearch.spark.rdd.AbstractEsRDDIterator.hasNext(AbstractEsRDDIterator.scala:61)
   at scala.collection.Iterator$anon$11.hasNext(Iterator.scala:408)
   at scala.collection.Iterator$anon$11.hasNext(Iterator.scala:408)
   at scala.collection.Iterator$anon$14.hasNext(Iterator.scala:533)
   at org.apache.spark.storage.memory.MemoryStore.putIteratorAsValues(MemoryStore.scala:216)
   at org.apache.spark.storage.BlockManager$anonfun$doPutIterator$1.apply(BlockManager.scala:1092)
   at org.apache.spark.storage.BlockManager$anonfun$doPutIterator$1.apply(BlockManager.scala:1083)
   at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:1018)
   at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1083)
   at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:809)
   at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:347)
   at org.apache.spark.rdd.RDD.iterator(RDD.scala:298)
   at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:42)
   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:336)
   at org.apache.spark.rdd.RDD.iterator(RDD.scala:300)
   at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
   at org.apache.spark.scheduler.Task.run(Task.scala:112)
   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:384)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   at java.lang.Thread.run(Thread.java:748)

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 3, 2019, 3:43pm UTC](https://discuss.elastic.co/t/spark-task-are-failing-no-search-context-found-for-id/163085/2 "2019-02-03T15:43:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.