# Basic Authentication with Spark fails with 403(forbidden)

**URL:** <https://discuss.elastic.co/t/basic-authentication-with-spark-fails-with-403-forbidden/38343>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [January 4, 2016, 7:43pm UTC](https://discuss.elastic.co/t/basic-authentication-with-spark-fails-with-403-forbidden/38343 "2016-01-04T19:43:41Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jvaradhan](https://avatars.discourse-cdn.com/v4/letter/j/9f8e36/32.png) [@jvaradhan](https://discuss.elastic.co/u/jvaradhan)\
**Post date:** [January 4, 2016, 7:43pm UTC](https://discuss.elastic.co/t/basic-authentication-with-spark-fails-with-403-forbidden/38343/1 "2016-01-04T19:43:41Z")

</div>

I am trying to write to ES with Basic auth. It fails with this error (see below). I tried it with straight java and other tools with setting the basic auth headers and they all work fine. Here is the code

```
val sparkConf = new SparkConf().setAppName("ElasticSearchTest").setMaster("local[*]")

sparkConf.set("es.nodes", "<elastic cluster host>")    
sparkConf.set("es.port", "9200")
sparkConf.set("es.net.http.auth.user", "Jack")
sparkConf.set("es.net.http.auth.pass", "1111111111111111")

val sc = new SparkContext(sparkConf)  

val json1 = """{"title" : "test"}"""      

sc.makeRDD(Seq(json1)).saveJsonToEs("a/b")  

```

```auto
Error:
org.elasticsearch.hadoop.rest.EsHadoopInvalidRequest: [HEAD] on [a] failed; server[xxx.xxx.xxx.xxx:9202] returned [403|Forbidden:]
	at org.elasticsearch.hadoop.rest.RestClient.checkResponse(RestClient.java:368)
	at org.elasticsearch.hadoop.rest.RestClient.executeNotFoundAllowed(RestClient.java:344)
	at org.elasticsearch.hadoop.rest.RestClient.exists(RestClient.java:409)
	at org.elasticsearch.hadoop.rest.RestClient.touch(RestClient.java:415)
	at org.elasticsearch.hadoop.rest.RestRepository.touch(RestRepository.java:443)
	at org.elasticsearch.hadoop.rest.RestService.initSingleIndex(RestService.java:408)
	at org.elasticsearch.hadoop.rest.RestService.createWriter(RestService.java:396)
	at org.elasticsearch.spark.rdd.EsRDDWriter.write(EsRDDWriter.scala:40)
	at org.elasticsearch.spark.rdd.EsSpark$$anonfun$saveToEs$1.apply(EsSpark.scala:67)
	at org.elasticsearch.spark.rdd.EsSpark$$anonfun$saveToEs$1.apply(EsSpark.scala:67)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
	at org.apache.spark.scheduler.Task.run(Task.scala:88)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

```

---

<div class="post-metadata">

**Author:** ![jvaradhan](https://avatars.discourse-cdn.com/v4/letter/j/9f8e36/32.png) [@jvaradhan](https://discuss.elastic.co/u/jvaradhan)\
**Post date:** [January 4, 2016, 7:45pm UTC](https://discuss.elastic.co/t/basic-authentication-with-spark-fails-with-403-forbidden/38343/2 "2016-01-04T19:45:10Z")

</div>

btw, I am using 2.1.2 version on elasticsearch-spark

compile 'org.elasticsearch:elasticsearch-spark\_2.10:2.1.2'

---

<div class="post-metadata">

**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\
**Post date:** [January 9, 2016, 1:23pm UTC](https://discuss.elastic.co/t/basic-authentication-with-spark-fails-with-403-forbidden/38343/3 "2016-01-09T13:23:10Z")

</div>

If the page on [security](https://www.elastic.co/guide/en/elasticsearch/hadoop/master/security.html) does not help, to diagnose the issue is best to turn on [logging](https://www.elastic.co/guide/en/elasticsearch/hadoop/master/logging.html) in particular on the REST and Spark packages.  
This provides info on the network traffic so one can see where the connection is being made and what information is passed through.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:26pm UTC](https://discuss.elastic.co/t/basic-authentication-with-spark-fails-with-403-forbidden/38343/4 "2017-07-06T13:26:40Z")

</div>