# Behaviour of the Java Spark API : JavaEsSpark.saveToEs (background or not)

**URL:** <https://discuss.elastic.co/t/behaviour-of-the-java-spark-api-javaesspark-savetoes-background-or-not/220215>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [February 20, 2020, 3:26pm UTC](https://discuss.elastic.co/t/behaviour-of-the-java-spark-api-javaesspark-savetoes-background-or-not/220215 "2020-02-20T15:26:54Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![sebastienf](https://avatars.discourse-cdn.com/v4/letter/s/f07891/32.png) [@sebastienf](https://discuss.elastic.co/u/sebastienf)\
**Post date:** [February 20, 2020, 3:26pm UTC](https://discuss.elastic.co/t/behaviour-of-the-java-spark-api-javaesspark-savetoes-background-or-not/220215/1 "2020-02-20T15:26:54Z")

</div>

Hi,  
In a Spark Streaming Job, I am using JavaEsSpark.saveToEs to bulk documents into ES indices.  
So far so good, but I was wondering about its behaviour.  
Indeed, I noticed that the code after the JavaEsSpark call seems to be executed whereas it is not over...

```
    if (...) {
               
      JavaEsSpark.saveToEs(message, index_pattern+"_{target_index}/{target_type}");
      
    }

    // at last, we commit the offset ranges
    ((CanCommitOffsets) messages.inputDStream()).commitAsync(offsetRanges);

```

As you can see, I commit kafka offset after saveToEs, but sometimes, saveToEs failed (what so ever issue) and offsets are still commited !

So, if anyone knows sharply the behaviour of this API, I would be pleased 🙂

Thanks a lot.

---

<div class="post-metadata">

**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\
**Post date:** [February 27, 2020, 4:28pm UTC](https://discuss.elastic.co/t/behaviour-of-the-java-spark-api-javaesspark-savetoes-background-or-not/220215/2 "2020-02-27T16:28:09Z")

</div>

That's pretty peculiar. The saveToEs method simply runs the spark job using the writer implementation to save the data to Elasticsearch. If the RDD fails to complete, the job runner should throw a `org.apache.spark.SparkException` with information about what caused the run to fail. Can you share what kind of failure you're seeing that still leads to the offsets being committed? Are you sure that the line of code here that commits the offset is the only place that the commit happens?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 26, 2020, 4:28pm UTC](https://discuss.elastic.co/t/behaviour-of-the-java-spark-api-javaesspark-savetoes-background-or-not/220215/3 "2020-03-26T16:28:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.