# Connecting to Elasticsearch node in Elastic cloud through python promt

**URL:** <https://discuss.elastic.co/t/connecting-to-elasticsearch-node-in-elastic-cloud-through-python-promt/289028>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-security, language-clients\
**Created:** [November 11, 2021, 10:24pm UTC](https://discuss.elastic.co/t/connecting-to-elasticsearch-node-in-elastic-cloud-through-python-promt/289028 "2021-11-11T22:24:19Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sentel\_AI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sentel_ai/32/97043_2.png) [@Sentel\_AI](https://discuss.elastic.co/u/Sentel_AI)\
**Post date:** [November 11, 2021, 10:24pm UTC](https://discuss.elastic.co/t/connecting-to-elasticsearch-node-in-elastic-cloud-through-python-promt/289028/1 "2021-11-11T22:24:19Z")

</div>

Hi, In order to create an ingestion pipeline into the Elasticsearch node running in Elastic Cloud, I'm using the following code (API authentication):

```auto
# API authentication
config = configparser.ConfigParser()
config.read('api_auth.ini')

es = Elasticsearch(
    cloud_id=config['DEFAULT']['cloud_id'],
    api_key=(config['DEFAULT']['apikey_id'], config['DEFAULT']['apikey_key']),
)

es.info()

```

where my api\_auth.ini file looks like:

```auto
[DEFAULT]
cloud_id = YYYYYYYYYYYYYYYYYY
apikey_id = ZZZZZZZZZZZZZZZZZ
apikey_key = XXXXXXXXXXXXXXX

```

However, I'm not being able to setup a connection and getting the following connection error:

```auto
elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7ff370053ee0>: Failed to establish a new connection: [Errno 61] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7ff370053ee0>: Failed to establish a new connection: [Errno 61] Connection refused)

```

Can you kindly help fixing this issue? Thanks.

---

<div class="post-metadata">

**Author:** ![Sentel\_AI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sentel_ai/32/97043_2.png) [@Sentel\_AI](https://discuss.elastic.co/u/Sentel_AI)\
**Post date:** [November 14, 2021, 10:40pm UTC](https://discuss.elastic.co/t/connecting-to-elasticsearch-node-in-elastic-cloud-through-python-promt/289028/2 "2021-11-14T22:40:49Z")

</div>

Adding `port=9200, scheme="https"` in the `es` now apparently fixed that error. But I'm getting the following connection error now:

```auto
elasticsearch.exceptions.ConnectionError: ConnectionError((<urllib3.connection.HTTPSConnection object at 0x7f7b98ecfc40>, 'Connection to 318ee65a637a4d979a1fb8f38045448c.us-central1.gcp.cloud.es.io timed out. (connect timeout=10)')) caused by: ConnectTimeoutError((<urllib3.connection.HTTPSConnection object at 0x7f7b98ecfc40>, 'Connection to 318ee65a637a4d979a1fb8f38045448c.us-central1.gcp.cloud.es.io timed out. (connect timeout=10)'))

```

Can anyone help fixing the connection timeout error? I've tried the following fix but didn't work out:

```auto
es.cluster.health(wait_for_status='yellow', request_timeout=1)

```

Anybody encountered similar error before?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [November 14, 2021, 11:45pm UTC](https://discuss.elastic.co/t/connecting-to-elasticsearch-node-in-elastic-cloud-through-python-promt/289028/3 "2021-11-14T23:45:12Z")

</div>

Bhi @Sentel_AI Welcome to the community!

I'm not a python guru but Elastic Cloud does not run on port 9200 it runs on 443 and/or 9243.

But according to [the docs](https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/connecting.html) you shouldn't need any of those are you sure you have the API key and ID correct and cloud ID correct?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [November 15, 2021, 1:10am UTC](https://discuss.elastic.co/t/connecting-to-elasticsearch-node-in-elastic-cloud-through-python-promt/289028/4 "2021-11-15T01:10:32Z")

</div>

I just did this

```auto
from elasticsearch import Elasticsearch

# you can use RFC-1738 to specify the url

# ... or specify common parameters as kwargs

es = Elasticsearch(
    cloud_id="mycluster:sdfgsdfgdsfgdsfglvdsfgsdfgdsfgmNiM2JkNzRjNDY3JGIxZTUyOWEwNTBkNjRkODZhMzIxZTBhMjU3YjRlODhh",
    http_auth=("elastic", "dsfgsdfgsdfgg"),
)

output = es.info(),

print(output, end=" ")

```

and it worked fine...

This also worked fine...

```auto
es = Elasticsearch(
    cloud_id="mycluster:sdfgsdfgdsfgdsfgsdfgsdfgsdfgsdfgsdfgsdfgsdfgsdfgdsfgkODZhMzIxZTBhMjU3YjRlODhh",
    api_key=('cew-IX0BKn301Oe9gPyz', '02I9MJ4OR2GdxYqY3I3cKA'),

```

remember this is NOT and Elastic Cloud API key it is an Elasticsearch API key which can only be created via and API call see [here](https://www.elastic.co/guide/en/elasticsearch/reference/7.15/security-api-create-api-key.html)

```auto
POST /_security/api_key
{
  "name": "my-api-key",
  "expiration": "1d",   
  "role_descriptors": { 
    "role-a": {
      "cluster": ["all"]
      
    }
  }
}

```

results

```auto
{
  "id" : "cew-IX0BKn301Oe9gPyz",
  "name" : "my-api-key",
  "expiration" : 1637026684066,
  "api_key" : "02I9MJ4OR2GdxYqY3I3cKA"
}

```

Please don't anyone worry those are invalid API keys

---

<div class="post-metadata">

**Author:** ![Sentel\_AI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sentel_ai/32/97043_2.png) [@Sentel\_AI](https://discuss.elastic.co/u/Sentel_AI)\
**Post date:** [November 15, 2021, 10:59am UTC](https://discuss.elastic.co/t/connecting-to-elasticsearch-node-in-elastic-cloud-through-python-promt/289028/5 "2021-11-15T10:59:13Z")

</div>

Thanks for the reply. I indeed created the Elastic Cloud authentication keys using the Elastic cloud documentation. Nonetheless, I just discovered that by directly passing the `cloud_id` and `http_auth` into the script at least fixes the connection issue. However, I'm still having ConnectionError when I'm trying to ingest some data.

```auto
elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7f98913b6c10>: Failed to establish a new connection: [Errno 61] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7f98913b6c10>: Failed to establish a new connection: [Errno 61] Connection refused)

```

Let me get back to you after a bit more debugging. Maybe the way I'm trying to write the data into the Elasticsearch node is problematic. Much thanks.

---

<div class="post-metadata">

**Author:** ![Sentel\_AI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sentel_ai/32/97043_2.png) [@Sentel\_AI](https://discuss.elastic.co/u/Sentel_AI)\
**Post date:** [November 15, 2021, 8:23pm UTC](https://discuss.elastic.co/t/connecting-to-elasticsearch-node-in-elastic-cloud-through-python-promt/289028/6 "2021-11-15T20:23:30Z")

</div>

Update: the problem somehow was in Espandas - the step where I am trying to ingest a pandas DataFrame into Elasticsearch. Nonetheless, for now I'm using an explicit mapping (i.e. by explicitly defining the mapping of the document) and only calling the functions from Elasticsearch-py to create and populate the indices. It works.

However, while using API authentication I did encounter the following error:

```auto
elasticsearch.exceptions.AuthorizationException: AuthorizationException(403, 'security_exception', 'action [indices:admin/create] is unauthorized for API key id [XXXXXXXXYYYYYZZZZZ] of user [found-internal-userconsole-proxy], this action is granted by the index privileges [create_index,manage,all]')

```

Nevertheless, for now, I'm using `elastic` user and being able to ingest the data (without using Espandas). I needed the solution quite fast. So this works for the time being. However, I would like to have an access with the API authentication as well. Will look into this further.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [November 15, 2021, 8:54pm UTC](https://discuss.elastic.co/t/connecting-to-elasticsearch-node-in-elastic-cloud-through-python-promt/289028/7 "2021-11-15T20:54:35Z")

</div>

Yes you will need to learn about users and roles.

> **[User authorization | Elasticsearch Guide \[7.15\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/authorization.html)**

Then you will need to create the correct API key

> **[Create API key API | Elasticsearch Guide \[7.15\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html)**

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 13, 2021, 8:54pm UTC](https://discuss.elastic.co/t/connecting-to-elasticsearch-node-in-elastic-cloud-through-python-promt/289028/8 "2021-12-13T20:54:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.