What's Python "best practice" for security certificates with ES8?

[mrodent (Mike)]

I just set up ES 8.6.2 on my machine. This is a single-machine setup. In fact I'm upgrading from 7.10.2, see previous question.

I've managed to obtain a password for user "elastic"... this means I can get the "You know, for search" page in a browser.

I've worked out how to include the username and password in Postman, so I can do that using a RESTful URL thing.

I've also worked out how to use the Python 'requests' package, using requests.auth.HTTPBasicAuth. But I find that unless I include a keyword pair "'verify' : False" in the kwargs I get an error like this:

requests.exceptions.SSLError: HTTPSConnectionPool(host='localhost', port=9500): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1123)')))

NB I have deliberately configured ES 8.6.2 to use port 9500. Also I prefer to use the 'requests' package rather than the 'elasticsearch' "thin wrapper" package, for simplicity/visibility.

The thing is, I really understand nothing at all about "security layers" and "certificates" and "certificate authorities". Although I've tried to understand, this whole topic is a complete mystery to me, and I'm just using ES on a single machine, to reverse-index documents.

I have seen this other question from 2019, but it is obviously by someone who understands about security layers and certificates.

The above workaround (verify: False) by default produces a warning from the urllib3 library: "... Unverified HTTPS request is being made to host 'localhost'. Adding certificate verification is strongly advised... " (!). It's possible to suppress this warning. But bypassing everything like this feels a bit wrong.

I am not sure whether there is a real security issue here: anyone connecting to my machine for nefarious purposes wouldn't for example be able to know the ES server password.

But I would like to know what the recommended best practice is with ES 8 when using Python.

[DavidTurner]

The Python requests library docs indicate you should set verify='/path/to/certfile' to specify a path to a CA_BUNDLE file or directory with certificates of trusted CAs. If you're using the certificates that Elasticsearch auto-generated then I believe it will have stored the CA root cert in $ES_PATH_CONF/certs/http_ca.crt in a format that should work as the right CA_BUNDLE file.