Issue with writing Spark dataset to Elastic

[angryninja]

Hi,

I'm trying to write a Spark dataset to es but I get following error while executing following piece of code -
items.write().format("es").options(getOptions()).save();

I'm also setting following options while trying to write the dataset

        addOption(Constants.ELASTICSEARCH_HOST,host);
        addOption(Constants.ELASTICSEARCH_PORT, "9200");
        addOption(Constants.INDEX,index);

Error encountered
org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: resource must be specified for Elasticsearch resources.

How do I know what else is required for it to write the data / to specify the resource ?

[Luca_Belluccini]

The documentation for ES-Hadoop Spark is available on our documentation.

In particular, for Spark SQL Streaming, you can refer to this documentation section.

Options should be passed es.resource with value index/type (e.g. itemsindex/_doc).
Or you can specify it in the save().

Also ensure Constants.ELASTICSEARCH_HOST is actually es.nodes and Constants.ELASTICSEARCH_PORT is actually es.port.

All the options can be found here.

[angryninja]

Thank you !