# Connecting an ES client with a kibana on Python

**URL:** <https://discuss.elastic.co/t/connecting-an-es-client-with-a-kibana-on-python/370991>\
**Category:** Elastic Security\
**Created:** [November 24, 2024, 9:19am UTC](https://discuss.elastic.co/t/connecting-an-es-client-with-a-kibana-on-python/370991 "2024-11-24T09:19:12Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![ALXIReinar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alxireinar/32/139428_2.png) [@ALXIReinar](https://discuss.elastic.co/u/ALXIReinar)\
**Post date:** [November 24, 2024, 9:19am UTC](https://discuss.elastic.co/t/connecting-an-es-client-with-a-kibana-on-python/370991/1 "2024-11-24T09:19:12Z")

</div>

This is the problem: I need Kibana and the ES client to work on python at the same time.  
At the same time, all security settings are enabled:

```auto
xpack.security.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.http.ssl:
 enabled: true
xpack.security.transport.ssl:
 enabled: true

```

## **1. Standard Authentication**

I tried to do it the usual way, through authentication:

```python
aioes = AsyncElasticsearch(
    hosts='http://localhost:9200',
    basic_auth=("user", "password")
)

```

But the client cannot reach the host. Disconnect is happening.  
The 2nd security parameter is responsible for this. When it is disabled, the ES client works, but the kibana crashes. And vice versa

## **2. Generate Api Key**

I tried to solve this problem through the api key.

In kibana, I made such a request:

```auto
POST _security/api_key
{
  "name": "ES_client",
  "role_descriptors": {
    "es_python_client": {
      "cluster": ["all"],
      "index": [
        {
          "names": ["index_timestamp", "dish_info_index", "test-index"],
          "privileges": ["all"]
        }
      ]
    }
  }
}

```

After receiving the api, I went to add it to the elastic. I typed in this command:

```auto
root: elasticsearch-8.15.4
$ ./bin/elasticsearch-keystore add es_client.python.api_key

```

The key has been added

After receiving the api, I went to add it to the elastic. I typed in this command:

The key has been added

## **3. Delete api key from keystore**

But after that, my elasticsearch didn't start at all.  
Therefore, I was forced to delete the api key. I typed in this command:

```auto
./bin/elasticsearch-keystore remove es_client.python.api_key

```

The Elatsik is working

I do not know what I did wrong. I will be very glad if someone shows me a way to solve my problem 😢

The main thing is that:

1. Аuthorization on kibanа remains
2. At the same time, it was possible to work with both Kibana and the ES python client

Thanks!

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [November 24, 2024, 6:15pm UTC](https://discuss.elastic.co/t/connecting-an-es-client-with-a-kibana-on-python/370991/2 "2024-11-24T18:15:36Z")

</div>

Hi @ALXIReinar Welcome to the community

> [@ALXIReinar](#):
>
> ```auto
> aioes = AsyncElasticsearch(
> hosts='http://localhost:9200',
> basic_auth=("user", "password")
> )
> 
> ```

since you enabled

```auto
xpack.security.http.ssl:
 enabled: true

```

the connection `hosts` needs to be `https`

You should refer to:

> **[Connecting | Elasticsearch Python Client \[8.15\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/client/python-api/8.15/connecting.html#_verifying_https_with_ca_certificates)**

You will need the CA as well as shown in that example

```auto
# Create the client instance
client = Elasticsearch(
    "https://localhost:9200",
    ca_certs="/path/to/http_ca.crt",
    api_key="api_key",
)

```

Please take a close look at the documentation.

[https://elasticsearch-py.readthedocs.io/en/v8.15.0/api/elasticsearch.html](https://elasticsearch-py.readthedocs.io/en/v8.15.0/api/elasticsearch.html)

I am not sure why you are doing this... this is not needed... the Python client does not read from the elastic keystore... you do not need to put the API key you generated in there.

> [@ALXIReinar](#):
>
> ```auto
> root: elasticsearch-8.15.4
> $ ./bin/elasticsearch-keystore add es_client.python.api_key
> 
> ```

The basic steps are

Generate the API Key from Kibana

Connect using the https endpoint + API Key + the CA Cert

You could also test with this to not validate the cert, but that is not recommended for production.

```auto
# Create the client instance
client = Elasticsearch(
    "https://localhost:9200",
    verify_certs=false,
    api_key="api_key",
)

```

---

<div class="post-metadata">

**Author:** ![ALXIReinar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alxireinar/32/139428_2.png) [@ALXIReinar](https://discuss.elastic.co/u/ALXIReinar)\
**Post date:** [November 24, 2024, 7:20pm UTC](https://discuss.elastic.co/t/connecting-an-es-client-with-a-kibana-on-python/370991/3 "2024-11-24T19:20:23Z")

</div>

Thank you very much, Stephen! I had assumptions about https for this case, but I wasn't completely sure about it.  
Anyway, thanks a lot again!

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [November 24, 2024, 7:23pm UTC](https://discuss.elastic.co/t/connecting-an-es-client-with-a-kibana-on-python/370991/4 "2024-11-24T19:23:48Z")

</div>

Did you get it to connect?

---

<div class="post-metadata">

**Author:** ![ALXIReinar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alxireinar/32/139428_2.png) [@ALXIReinar](https://discuss.elastic.co/u/ALXIReinar)\
**Post date:** [November 24, 2024, 8:04pm UTC](https://discuss.elastic.co/t/connecting-an-es-client-with-a-kibana-on-python/370991/5 "2024-11-24T20:04:31Z")

</div>

I haven't tried it yet. I decided that tomorrow I would sort out this headache with a fresh head. I have only recently started studying this wonderful search engine, so there are some difficulties with using it. I will be happy to report the result:)

---

<div class="post-metadata">

**Author:** ![ALXIReinar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alxireinar/32/139428_2.png) [@ALXIReinar](https://discuss.elastic.co/u/ALXIReinar)\
**Post date:** [November 25, 2024, 9:10pm UTC](https://discuss.elastic.co/t/connecting-an-es-client-with-a-kibana-on-python/370991/6 "2024-11-25T21:10:55Z")

</div>

Alas, I was unable to connect 🥲

**Code**

```auto
aioes = AsyncElasticsearch(
    hosts='https://localhost:9200',
    api_key='API',
    ca_certs='/config/certs/http_ca.crt'
)

```

**Output**  
`ValueError: ca_certs parameter is not a path`

It did not work either through the Api key or with basic authorization  
I assumed it was about asynchrony, but even here it's past:

**Code**

```auto
es = Elasticsearch(
    hosts='https://localhost:9200',
    basic_auth=('elastic', 'password'),
    ca_certs='/config/certs/http_ca.crt'
)

res = es.search(index='index_index')
print(res)

```

**Output**  
`elastic_transport.TlsError: TLS error caused by: TlsError(TLS error caused by: SSLError([Errno 2] No such file or directory))`

However, in this case, it already writes that the path to the file is incorrect... Here I stopped understanding even more what my mistake was. 😅

`elasticsearch-8.15.4\config\certs` - the PATH to the CA

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/a/3a34add272d4f3160a23d8334d56445369d4d3b2.png)

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [November 25, 2024, 10:52pm UTC](https://discuss.elastic.co/t/connecting-an-es-client-with-a-kibana-on-python/370991/7 "2024-11-25T22:52:44Z")

</div>

```auto
es = AsyncElasticsearch(
    hosts='https://localhost:9200',
    api_key='API',
    ca_certs='/config/certs/http_ca.crt'
)

```

You need to put the full path to the certs that were created from the root directory.

Like

`ca_certs=/Users/stephen/elasticsearch-8.15.4/config/certs/http_ca.crt`

If it's Windows you need the full path as well!

---

<div class="post-metadata">

**Author:** ![ALXIReinar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alxireinar/32/139428_2.png) [@ALXIReinar](https://discuss.elastic.co/u/ALXIReinar)\
**Post date:** [November 26, 2024, 3:22pm UTC](https://discuss.elastic.co/t/connecting-an-es-client-with-a-kibana-on-python/370991/8 "2024-11-26T15:22:20Z")

</div>

## Many thanks

Oh, Stephen... Everything worked out!!! Finally!  
I've been in agony since these 6 days. I have constant problems with ensuring that the middleware for my projects is working properly.  
The funny thing about this story is that yesterday I tried to set up security (yes, it was already set up automatically without me). Everything broke down for me, so I had to put elatsik and kibana on a new one. I lost 4.5 hours on this. This is terrible

I am grateful for your help

## The final version

With safety turned on in the elasticsearch.yml:

```auto
# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12

```

In my case, this setting worked:

```auto
es = AsyncElasticsearch(
    hosts='https://localhost:9200',
    basic_auth=('elastic', 'password'),
    ca_certs='C:/Users/User/Desktop/elasticsearch-8.15.4/config/certs/http_ca.crt'
)

```

The same is true for the synchronous version of the ES client

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 24, 2024, 3:23pm UTC](https://discuss.elastic.co/t/connecting-an-es-client-with-a-kibana-on-python/370991/9 "2024-12-24T15:23:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.