Noobish Elasticsearch Python question

Hello,

Since corona-wise, we got to stay at home nowadays, I was thinking it's a good time to learn a bit about the Elastic Python client library. First of all, apologies for the noobish question..

I seem to have difficulties connecting to an Elasticsearch node. My goal is very simple (for now) => To cat all indices on a cluster.

First I tried:

from elasticsearch import Elasticsearch
if __name__ == '__main__': 
    es = Elasticsearch(
        ['<myesnode>'],
        http_auth=('elastic', '<pass>'),
        scheme="https",
        port=9200,
    )
    print(es2.cat.indices("index_name", h=("h","s","i","id","p","r","dc","dd","ss","creation.date.string"), s="creation.date"))

But I get a connection error:

urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x04DFD3B8>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed

Then I tried without kwargs:

es2=Elasticsearch(['https://elastic:<pass>@<myesnode>:9200/']) 
print(es2.cat.indices("index_name", h=("h","s","i","id","p","r","dc","dd","ss","creation.date.string"), s="creation.date"))

And then I get:

elasticsearch.exceptions.SSLError: ConnectionError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)) caused by: SSLError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108))

The ssl certificate for our clusters are created with a private pki. The CA is in the Windows certificate store, so should be recognised?

I have exported our CA chain pem to a location on my C drive, but how can I make the es connection work and use a supplied ca pem?

Thanks!

WIllem

You could try extending the default Connection class and passing your SSLContext to it (with the right CA file). See the docs: https://elasticsearch-py.readthedocs.io/en/master/connection.html#urllib3httpconnection-default-connection-class

Then you can pass it to Elasticsearch using Elasticsearch(connection_class=MyConnectionClass)

Thanks @aramperes

So not sure what to do with context, do I need to pass it to connection_class?. Do you mean:

from elasticsearch import Elasticsearch
from ssl import create_default_context
context = create_default_context(cadata='<myca>')
from elasticsearch import Elasticsearch
if __name__ == '__main__': 
    es = Elasticsearch(
        ['<myesnode>'],
        http_auth=('elastic', '<pass>'),
        scheme="https",
        port=9200,
        connection_class=context
    )
    print(es2.cat.indices("index_name", h=("h","s","i","id","p","r","dc","dd","ss","creation.date.string"), s="creation.date"))

Unfortunately that ends with:

TypeError: 'SSLContext' object is not callable

Tried this:

from elasticsearch import Elasticsearch
from ssl import create_default_context
context = create_default_context(cadata='<myca>')
from elasticsearch import Elasticsearch
if __name__ == '__main__': 
    es = Elasticsearch(
        ['<myesnode>'],
        http_auth=('elastic', '<pass>'),
        scheme="https",
        port=9200,
        ca_certs='C:\Windows\PKI\MYCACHAIN.pem'
    )
    print(es2.cat.indices("index_name", h=("h","s","i","id","p","r","dc","dd","ss","creation.date.string"), s="creation.date"))

elasticsearch.exceptions.SSLError: ConnectionError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)) caused by: SSLError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108))

Which doesn't get me that much further.. :slight_smile:

Seting verify_certs=False results in the same error.

The error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate strongly indicates that you are supplying a CA chain that doesn't match the certificate from the server, possibly because you are passing an intermediate certificate instead.

You could try to perform the SSL handshake with OpenSSL and see if you get similar errors:

openssl s_client -connect <server>:9200 -CAfile </path/to/ca>

Ok, migrated the script to a CentOS server. The openssl check throws a lot of confidential info, but it seems ok, first line says "CONNECTED(00000003)"

So using this now:

#!/usr/bin/python
from elasticsearch import Elasticsearch, RequestsHttpConnection
from ssl import create_default_context
context = create_default_context(cafile='/etc/pki/ca-trust/source/anchors/SECURECHAIN.pem')
if __name__ == '__main__':
    es = Elasticsearch(
        ['<es-node>'],
        http_auth=('elastic', '<pw>'),
        scheme="https",
        port=9200,
        ca_certs='/etc/pki/ca-trust/source/anchors/SECURECHAIN.pem',
        connection_class=context
    )
    print(es.cat.indices("index_name", h=("h","s","i","id","p","r","dc","dd","ss","creation.date.string"), s="creation.date"))

Which also throws: Error: 'SSLContext' object is not callable

Sorry, I misread the documentation myself :smile: What if you try Elasticsearch(ssl_context=context)?

Np, so I'm using this now:

#!/usr/bin/python
from elasticsearch import Elasticsearch, RequestsHttpConnection
from ssl import create_default_context
context = create_default_context(cafile='/etc/pki/ca-trust/source/anchors/SECURECHAIN.pem')
if __name__ == '__main__':
    es = Elasticsearch(
        ['<es-node>'],
        http_auth=('elastic', '<pw>'),
        scheme="https",
        port=9200,
        ssl_context=context
    )
    print(es.cat.indices("index_name", h=("h","s","i","id","p","r","dc","dd","ss","creation.date.string"), s="creation.date"))

Throws me:

Traceback (most recent call last):
  File "./check_elasticsearch_shard.py", line 16, in <module>
    print(es.cat.indices("index_name", h=("h","s","i","id","p","r","dc","dd","ss","creation.date.string"), s="creation.date"))
  File "/usr/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 92, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
  File "/usr/lib/python2.7/site-packages/elasticsearch/client/cat.py", line 161, in indices
    "GET", _make_path("_cat", "indices", index), params=params, headers=headers
  File "/usr/lib/python2.7/site-packages/elasticsearch/transport.py", line 362, in perform_request
    timeout=timeout,
  File "/usr/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 241, in perform_request
    raise ConnectionError("N/A", str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7f3e8b8af490>: Failed to establish a new connection: [Errno -2] Name or service not known) caused by: NewConnectionError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7f3e8b8af490>: Failed to establish a new connection: [Errno -2] Name or service not known)

That error is unrelated to SSL; at that point, it's a DNS issue (the hostname you provided might be wrong).

How about you ditch ssl_context altogether and just use ca_certs?

If you can reproduce that the openssl command I provided works with the same CA that fails on the Python library, I'm not sure what the problem is. Make sure your script is talking to the right port. The key thing to check with OpenSSL is that the server certificate was validated properly with the given CA. Make sure the handshake looks like it works.

Also you used a different paths in your posts, .../SECURECHAIN.pem and .../GENTSECURECHAIN.pem. Typo?

Definitely no typo, just some bad obfuscation. Please remove the first pem from your post. I'll investigate further today.

@aramperes Ok, managed to make it work. Several issues caused this.

  • Needed to start VS Code as admin to be able to read my pem on WIndows
  • Needed to use \\ in the path of cafile
  • Can't mix context with ssl settings in the Elasticsearch object

Thanks for the help and guidance!

Hello,

Found one more reason why my script did not work on 1 cluster. The password for the user I'm using to connect to this 1 cluster ended with a '@'. It seems using this particular user always results in:

elasticsearch.exceptions.ConnectionError: ConnectionError(('Connection aborted.', OSError(0, 'Error'))) caused by: ProtocolError(('Connection aborted.', OSError(0, 'Error')))

So is this a bug or is there a way to prevent my script from failing when using:

es = Elasticsearch(
        [str(es_host)],
        http_auth = (str(es_user), str(es_pass)),
        scheme = "https",
        port = int(es_port),
        ssl_context = context
    )

Grtz

Willem

Sry, it seems like I was wrong. The particular problematic cluster had the wrong http ip filters. Due to us having to work from home, some vpn ip ranges hadn't been added....