# Issue with writing Spark dataset to Elastic

**URL:** <https://discuss.elastic.co/t/issue-with-writing-spark-dataset-to-elastic/229694>\
**Category:** Elasticsearch\
**Created:** [April 24, 2020, 4:31pm UTC](https://discuss.elastic.co/t/issue-with-writing-spark-dataset-to-elastic/229694 "2020-04-24T16:31:51Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![angryninja](https://avatars.discourse-cdn.com/v4/letter/a/59ef9b/32.png) [@angryninja](https://discuss.elastic.co/u/angryninja)\
**Post date:** [April 24, 2020, 4:31pm UTC](https://discuss.elastic.co/t/issue-with-writing-spark-dataset-to-elastic/229694/1 "2020-04-24T16:31:51Z")

</div>

Hi,

I'm trying to write a Spark dataset to es but I get following error while executing following piece of code -  
`items.write().format("es").options(getOptions()).save();`

I'm also setting following options while trying to write the dataset

```auto
        addOption(Constants.ELASTICSEARCH_HOST,host);
        addOption(Constants.ELASTICSEARCH_PORT, "9200");
        addOption(Constants.INDEX,index);

```

Error encountered  
`org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: resource must be specified for Elasticsearch resources.`

How do I know what else is required for it to write the data / to specify the resource ?

---

<div class="post-metadata">

**Author:** ![Luca\_Belluccini](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/luca_belluccini/32/33239_2.png) [@Luca\_Belluccini](https://discuss.elastic.co/u/Luca_Belluccini)\
**Post date:** [April 24, 2020, 5:05pm UTC](https://discuss.elastic.co/t/issue-with-writing-spark-dataset-to-elastic/229694/2 "2020-04-24T17:05:22Z")

</div>

The documentation for ES-Hadoop Spark is available [on our documentation](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/spark.html).

In particular, for Spark SQL Streaming, you can refer to this [documentation section](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/spark.html#spark-sql-streaming-write-scala).

Options should be passed `es.resource` with value `index/type` (e.g. `itemsindex/_doc`).  
Or you can specify it in the `save()`.

Also ensure `Constants.ELASTICSEARCH_HOST` is actually `es.nodes` and `Constants.ELASTICSEARCH_PORT` is actually `es.port`.

All the options can be found [here](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html#_required_settings).

---

<div class="post-metadata">

**Author:** ![angryninja](https://avatars.discourse-cdn.com/v4/letter/a/59ef9b/32.png) [@angryninja](https://discuss.elastic.co/u/angryninja)\
**Post date:** [April 24, 2020, 7:12pm UTC](https://discuss.elastic.co/t/issue-with-writing-spark-dataset-to-elastic/229694/3 "2020-04-24T19:12:56Z")

</div>

Thank you !

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 22, 2020, 7:12pm UTC](https://discuss.elastic.co/t/issue-with-writing-spark-dataset-to-elastic/229694/4 "2020-05-22T19:12:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.