Noobish Elasticsearch Python question

willemdh · March 20, 2020, 3:42pm

Hello,

Since corona-wise, we got to stay at home nowadays, I was thinking it's a good time to learn a bit about the Elastic Python client library. First of all, apologies for the noobish question..

I seem to have difficulties connecting to an Elasticsearch node. My goal is very simple (for now) => To cat all indices on a cluster.

First I tried:

from elasticsearch import Elasticsearch
if __name__ == '__main__': 
    es = Elasticsearch(
        ['<myesnode>'],
        http_auth=('elastic', '<pass>'),
        scheme="https",
        port=9200,
    )
    print(es2.cat.indices("index_name", h=("h","s","i","id","p","r","dc","dd","ss","creation.date.string"), s="creation.date"))

But I get a connection error:

urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x04DFD3B8>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed

Then I tried without kwargs:

es2=Elasticsearch(['https://elastic:<pass>@<myesnode>:9200/']) 
print(es2.cat.indices("index_name", h=("h","s","i","id","p","r","dc","dd","ss","creation.date.string"), s="creation.date"))

And then I get:

elasticsearch.exceptions.SSLError: ConnectionError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)) caused by: SSLError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108))

The ssl certificate for our clusters are created with a private pki. The CA is in the Windows certificate store, so should be recognised?

I have exported our CA chain pem to a location on my C drive, but how can I make the es connection work and use a supplied ca pem?

Thanks!

WIllem

aramperes · March 20, 2020, 4:10pm

You could try extending the default Connection class and passing your SSLContext to it (with the right CA file). See the docs: https://elasticsearch-py.readthedocs.io/en/master/connection.html#urllib3httpconnection-default-connection-class

Then you can pass it to Elasticsearch using Elasticsearch(connection_class=MyConnectionClass)

willemdh · March 20, 2020, 5:00pm

Thanks @aramperes

So not sure what to do with context, do I need to pass it to connection_class?. Do you mean:

from elasticsearch import Elasticsearch
from ssl import create_default_context
context = create_default_context(cadata='<myca>')
from elasticsearch import Elasticsearch
if __name__ == '__main__': 
    es = Elasticsearch(
        ['<myesnode>'],
        http_auth=('elastic', '<pass>'),
        scheme="https",
        port=9200,
        connection_class=context
    )
    print(es2.cat.indices("index_name", h=("h","s","i","id","p","r","dc","dd","ss","creation.date.string"), s="creation.date"))

Unfortunately that ends with:

TypeError: 'SSLContext' object is not callable

willemdh · March 20, 2020, 5:40pm

Tried this:

from elasticsearch import Elasticsearch
from ssl import create_default_context
context = create_default_context(cadata='<myca>')
from elasticsearch import Elasticsearch
if __name__ == '__main__': 
    es = Elasticsearch(
        ['<myesnode>'],
        http_auth=('elastic', '<pass>'),
        scheme="https",
        port=9200,
        ca_certs='C:\Windows\PKI\MYCACHAIN.pem'
    )
    print(es2.cat.indices("index_name", h=("h","s","i","id","p","r","dc","dd","ss","creation.date.string"), s="creation.date"))

elasticsearch.exceptions.SSLError: ConnectionError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)) caused by: SSLError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108))

Which doesn't get me that much further..

Seting verify_certs=False results in the same error.

aramperes · March 20, 2020, 6:38pm

The error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate strongly indicates that you are supplying a CA chain that doesn't match the certificate from the server, possibly because you are passing an intermediate certificate instead.

You could try to perform the SSL handshake with OpenSSL and see if you get similar errors:

openssl s_client -connect <server>:9200 -CAfile </path/to/ca>

willemdh · March 20, 2020, 7:45pm

Ok, migrated the script to a CentOS server. The openssl check throws a lot of confidential info, but it seems ok, first line says "CONNECTED(00000003)"

So using this now:

#!/usr/bin/python
from elasticsearch import Elasticsearch, RequestsHttpConnection
from ssl import create_default_context
context = create_default_context(cafile='/etc/pki/ca-trust/source/anchors/SECURECHAIN.pem')
if __name__ == '__main__':
    es = Elasticsearch(
        ['<es-node>'],
        http_auth=('elastic', '<pw>'),
        scheme="https",
        port=9200,
        ca_certs='/etc/pki/ca-trust/source/anchors/SECURECHAIN.pem',
        connection_class=context
    )
    print(es.cat.indices("index_name", h=("h","s","i","id","p","r","dc","dd","ss","creation.date.string"), s="creation.date"))

Which also throws: Error: 'SSLContext' object is not callable

aramperes · March 20, 2020, 7:54pm

Sorry, I misread the documentation myself What if you try Elasticsearch(ssl_context=context)?

willemdh · March 20, 2020, 7:58pm

Np, so I'm using this now:

#!/usr/bin/python
from elasticsearch import Elasticsearch, RequestsHttpConnection
from ssl import create_default_context
context = create_default_context(cafile='/etc/pki/ca-trust/source/anchors/SECURECHAIN.pem')
if __name__ == '__main__':
    es = Elasticsearch(
        ['<es-node>'],
        http_auth=('elastic', '<pw>'),
        scheme="https",
        port=9200,
        ssl_context=context
    )
    print(es.cat.indices("index_name", h=("h","s","i","id","p","r","dc","dd","ss","creation.date.string"), s="creation.date"))

Throws me:

Traceback (most recent call last):
  File "./check_elasticsearch_shard.py", line 16, in <module>
    print(es.cat.indices("index_name", h=("h","s","i","id","p","r","dc","dd","ss","creation.date.string"), s="creation.date"))
  File "/usr/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 92, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
  File "/usr/lib/python2.7/site-packages/elasticsearch/client/cat.py", line 161, in indices
    "GET", _make_path("_cat", "indices", index), params=params, headers=headers
  File "/usr/lib/python2.7/site-packages/elasticsearch/transport.py", line 362, in perform_request
    timeout=timeout,
  File "/usr/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 241, in perform_request
    raise ConnectionError("N/A", str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7f3e8b8af490>: Failed to establish a new connection: [Errno -2] Name or service not known) caused by: NewConnectionError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7f3e8b8af490>: Failed to establish a new connection: [Errno -2] Name or service not known)

aramperes · March 20, 2020, 10:19pm

That error is unrelated to SSL; at that point, it's a DNS issue (the hostname you provided might be wrong).

How about you ditch ssl_context altogether and just use ca_certs?

If you can reproduce that the openssl command I provided works with the same CA that fails on the Python library, I'm not sure what the problem is. Make sure your script is talking to the right port. The key thing to check with OpenSSL is that the server certificate was validated properly with the given CA. Make sure the handshake looks like it works.

Also you used a different paths in your posts, .../SECURECHAIN.pem and .../GENTSECURECHAIN.pem. Typo?

willemdh · March 21, 2020, 8:50am

Definitely no typo, just some bad obfuscation. Please remove the first pem from your post. I'll investigate further today.

willemdh · March 21, 2020, 10:57am

@aramperes Ok, managed to make it work. Several issues caused this.

Needed to start VS Code as admin to be able to read my pem on WIndows
Needed to use \\ in the path of cafile
Can't mix context with ssl settings in the Elasticsearch object

Thanks for the help and guidance!

willemdh · March 23, 2020, 10:22am

Hello,

Found one more reason why my script did not work on 1 cluster. The password for the user I'm using to connect to this 1 cluster ended with a '@'. It seems using this particular user always results in:

elasticsearch.exceptions.ConnectionError: ConnectionError(('Connection aborted.', OSError(0, 'Error'))) caused by: ProtocolError(('Connection aborted.', OSError(0, 'Error')))

So is this a bug or is there a way to prevent my script from failing when using:

es = Elasticsearch(
        [str(es_host)],
        http_auth = (str(es_user), str(es_pass)),
        scheme = "https",
        port = int(es_port),
        ssl_context = context
    )

Grtz

Willem

willemdh · March 23, 2020, 1:58pm

Sry, it seems like I was wrong. The particular problematic cluster had the wrong http ip filters. Due to us having to work from home, some vpn ip ranges hadn't been added....

system · April 20, 2020, 1:58pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Connecting to elastic Kibana	4	94	October 2, 2023
Unable to get local issuer certificate (_ssl.c:1007))) Elasticsearch elastic-stack-security , docker	2	204	August 7, 2024
Python SSL Error Elasticsearch	13	27634	August 14, 2019
Python 3.6 and SSL issues Elasticsearch	3	1517	June 26, 2019
Elastic Search Connection error Elasticsearch language-clients	6	16042	February 18, 2021

Noobish Elasticsearch Python question

Related topics