# Unable to connect to AWS hosted elasticsearch from spark hadoop with 403 forbidden error
**URL:** https://discuss.elastic.co/t/unable-to-connect-to-aws-hosted-elasticsearch-from-spark-hadoop-with-403-forbidden-error/119946
**Category:** Elasticsearch
**Created:** [February 15, 2018, 9:11am UTC](https://discuss.elastic.co/t/unable-to-connect-to-aws-hosted-elasticsearch-from-spark-hadoop-with-403-forbidden-error/119946 "2018-02-15T09:11:33Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@Sathish4k](https://discuss.elastic.co/u/Sathish4k)
#### Post date: [February 15, 2018, 9:11am UTC](https://discuss.elastic.co/t/unable-to-connect-to-aws-hosted-elasticsearch-from-spark-hadoop-with-403-forbidden-error/119946/1 "2018-02-15T09:11:33Z")
Im trying to connect to a AWS hosted elastic search and I am able to connect to the ES host, but getting a 403 forbidden error.
This is how I connect to ES from spark shell
```
val conf = new SparkConf()
conf.set("spark.es.nodes","https://URL.amazonaws.com")
conf.set("spark.es.port","PORT")
conf.set("spark.es.nodes.discovery","false")
conf.set("spark.es.nodes.client.only","false")
conf.set("spark.es.nodes.wan.only","true")
conf.set("spark.es.net.http.auth.user","AWS SECRET ID")
conf.set("spark.es.net.http.auth.pass","AWS SECRET PASS")
conf.set("spark.es.resouce.auto.create","XXX")
val sc = new SparkContext(conf)
val sqlContext = new org.apache.spark.sql.SQLContext(sc);
val numbers = Map("one" -> 1, "two" -> 2, "three" -> 3)
val airports = Map("arrival" -> "Otopeni", "SFO" -> "San Fran")
sc.makeRDD(Seq(numbers, airports)).saveToEs("tracking_events/events")
```
I am getting the below error
```
18/02/15 08:54:29 DEBUG content: << "{"message":"Authorization header requires 'Credential' parameter. Authorization header requires 'Signature' parameter. Authorization header requires 'SignedHeaders' parameter. Authorization header requires existence of either a 'X-Amz-Date' or a 'Date' header. Authorization=Basic <<<<<<<>>>>>>>>>=="}"
18/02/15 08:54:29 TRACE HeaderElement: enter HeaderElement.parseElements(char[])
18/02/15 08:54:29 TRACE HeaderElement: enter HeaderElement.parseElements(String)
18/02/15 08:54:29 TRACE HeaderElement: enter HeaderElement.parseElements(String)
18/02/15 08:54:29 TRACE HeaderElement: enter HeaderElement.parseElements(char[])
18/02/15 08:54:29 DEBUG HttpMethodBase: Should NOT close connection in response to directive: keep-alive
18/02/15 08:54:29 TRACE HttpConnection: enter HttpConnection.isResponseAvailable()
18/02/15 08:54:29 TRACE HeaderElement: enter HeaderElement.parseElements(char[])
18/02/15 08:54:29 TRACE HttpConnection: enter HttpConnection.releaseConnection()
18/02/15 08:54:29 DEBUG HttpConnection: Releasing connection back to connection manager.
18/02/15 08:54:29 TRACE HttpMethodBase: enter getContentCharSet( Header contentheader )
18/02/15 08:54:29 TRACE HeaderElement: enter HeaderElement.parseElements(String)
18/02/15 08:54:29 TRACE HeaderElement: enter HeaderElement.parseElements(char[])
18/02/15 08:54:29 TRACE HeaderElement: enter HeaderElement.getParameterByName(String)
18/02/15 08:54:29 TRACE HeaderElement: enter HeaderElement.getParameterByName(String)
18/02/15 08:54:29 TRACE HeaderElement: enter HeaderElement.getParameterByName(String)
18/02/15 08:54:29 TRACE HeaderElement: enter HeaderElement.getParameterByName(String)
18/02/15 08:54:29 DEBUG HttpMethodBase: Default charset used: UTF-8
18/02/15 08:54:29 DEBUG HttpMethodBase: Default charset used: UTF-8
18/02/15 08:54:29 DEBUG HttpMethodBase: Default charset used: UTF-8
18/02/15 08:54:29 DEBUG HttpMethodBase: Default charset used: UTF-8
18/02/15 08:54:29 TRACE CommonsHttpTransport: Rx @[x.x.x.x] [403-Forbidden] [{"message":"Authorization header requires 'Credential' parameter. Authorization header requires 'Signature' parameter. Authorization header requires 'SignedHeaders' parameter. Authorization header requires existence of either a 'X-Amz-Date' or a 'Date' header. Authorization=Basic <<<<<<<>>>>>>>>>=="}]
18/02/15 08:54:29 TRACE CommonsHttpTransport: Rx @[x.x.x.x] [403-Forbidden] [{"message":"Authorization header requires 'Credential' parameter. Authorization header requires 'Signature' parameter. Authorization header requires 'SignedHeaders' parameter. Authorization header requires existence of either a 'X-Amz-Date' or a 'Date' header. Authorization=Basic <<<<<<<>>>>>>>>>=="}]
18/02/15 08:54:29 TRACE CommonsHttpTransport: Rx @[x.x.x.x] [403-Forbidden] [{"message":"Authorization header requires 'Credential' parameter. Authorization header requires 'Signature' parameter. Authorization header requires 'SignedHeaders' parameter. Authorization header requires existence of either a 'X-Amz-Date' or a 'Date' header. Authorization=Basic <<<<<<<>>>>>>>>>=="}]
18/02/15 08:54:29 TRACE CommonsHttpTransport: Rx @[x.x.x.x] [403-Forbidden] [{"message":"Authorization header requires 'Credential' parameter. Authorization header requires 'Signature' parameter. Authorization header requires 'SignedHeaders' parameter. Authorization header requires existence of either a 'X-Amz-Date' or a 'Date' header. Authorization=Basic <<<<<<<>>>>>>>>>=="}]
18/02/15 08:54:29 DEBUG HttpMethodBase: re-creating response stream from byte array
18/02/15 08:54:29 DEBUG HttpMethodBase: re-creating response stream from byte array
18/02/15 08:54:29 DEBUG HttpMethodBase: re-creating response stream from byte array
18/02/15 08:54:29 DEBUG HttpMethodBase: re-creating response stream from byte array
18/02/15 08:54:29 DEBUG HttpMethodBase: re-creating response stream from byte array
18/02/15 08:54:29 DEBUG HttpMethodBase: re-creating response stream from byte array
18/02/15 08:54:29 DEBUG HttpMethodBase: re-creating response stream from byte array
18/02/15 08:54:29 DEBUG HttpMethodBase: re-creating response stream from byte array
18/02/15 08:54:29 TRACE CommonsHttpTransport: Closing HTTP transport to https://<<<<<<<>>>>>>>>>
18/02/15 08:54:29 TRACE HttpConnection: enter HttpConnection.close()
18/02/15 08:54:29 TRACE HttpConnection: enter HttpConnection.closeSockedAndStreams()
18/02/15 08:54:29 TRACE CommonsHttpTransport: Closing HTTP transport to https://<<<<<<<>>>>>>>>>
18/02/15 08:54:29 TRACE HttpConnection: enter HttpConnection.close()
18/02/15 08:54:29 TRACE HttpConnection: enter HttpConnection.closeSockedAndStreams()
18/02/15 08:54:29 TRACE CommonsHttpTransport: Closing HTTP transport to https://<<<<<<<>>>>>>>>>
18/02/15 08:54:29 TRACE HttpConnection: enter HttpConnection.close()
18/02/15 08:54:29 TRACE HttpConnection: enter HttpConnection.closeSockedAndStreams()
18/02/15 08:54:29 TRACE CommonsHttpTransport: Closing HTTP transport to https://<<<<<<<>>>>>>>>>
18/02/15 08:54:29 TRACE HttpConnection: enter HttpConnection.close()
18/02/15 08:54:29 TRACE HttpConnection: enter HttpConnection.closeSockedAndStreams()
18/02/15 08:54:29 ERROR Executor: Exception in task 1.0 in stage 0.0 (TID 1)
org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'
at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:190)
```
Is the authorization format wrong here?
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 15, 2018, 9:11am UTC](https://discuss.elastic.co/t/unable-to-connect-to-aws-hosted-elasticsearch-from-spark-hadoop-with-403-forbidden-error/119946/2 "2018-03-15T09:11:57Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.