Elasticsearch-py: how to hold session (python client)

Elastic Stack Elasticsearch
1

Hi All,
we have elasticsearch on cloud and various clients push data in seprate index.
and we are not using sheild, where as what we have is nginx implimented on public front.
and we are using auth + ssl to protect it.
with python elasticsearch client, things are working fine.
only issue now is, on each reqest we are performing authentication. So just wanted to know if we can hold the session some how.

Thanks in advance.

2

Wondering whether we have a solution to keep alive the session ?
It's expensive to keep doing auth .

Also the extra arguments passed are not being passed on to next set of calls.


 es = Elasticsearch(
     ['https://'+user+':'+passwd+'@'+host+':'+str(port)],
     ca_certs=ca_certs,
     verify_ssl=True,
 )
 es.search()


Traceback (most recent call last):
  File "test.py", line 37, in <module>
    verify_ssl=True,
  File "/usr/local/lib/python3.6/site-packages/elasticsearch/client/__init__.py", line 150, in __init__
    self.transport = transport_class(_normalize_hosts(hosts), **kwargs)
  File "/usr/local/lib/python3.6/site-packages/elasticsearch/transport.py", line 116, in __init__
    self.set_connections(hosts)
  File "/usr/local/lib/python3.6/site-packages/elasticsearch/transport.py", line 171, in set_connections
    connections = list(zip(connections, hosts))
  File "/usr/local/lib/python3.6/site-packages/elasticsearch/transport.py", line 168, in _create_connection
    return self.connection_class(**kwargs)
  File "/usr/local/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 47, in __init__
    raise ImproperlyConfigured("You cannot pass CA certificates when verify SSL is off.")
elasticsearch.exceptions.ImproperlyConfigured: You cannot pass CA certificates when verify SSL is off.

After upgrading to latest

$ pip3 install -r requirements.txt
Collecting elasticsearch>=1.9.0 (from -r requirements.txt (line 1))
  Using cached elasticsearch-5.3.0-py2.py3-none-any.whl
Requirement already satisfied: pyOpenSSL>=16.1.0 in /usr/local/lib/python3.6/site-packages (from -r requirements.txt (line 2))
Requirement already satisfied: keyring>=10.3.1 in /usr/local/lib/python3.6/site-packages (from -r requirements.txt (line 3))
Requirement already satisfied: urllib3<2.0,>=1.8 in /usr/local/lib/python3.6/site-packages (from elasticsearch>=1.9.0->-r requirements.txt (line 1))
Requirement already satisfied: cryptography>=1.3.4 in /usr/local/lib/python3.6/site-packages (from pyOpenSSL>=16.1.0->-r requirements.txt (line 2))
Requirement already satisfied: six>=1.5.2 in /usr/local/lib/python3.6/site-packages (from pyOpenSSL>=16.1.0->-r requirements.txt (line 2))
Requirement already satisfied: cffi>=1.4.1 in /usr/local/lib/python3.6/site-packages (from cryptography>=1.3.4->pyOpenSSL>=16.1.0->-r requirements.txt (line 2))
Requirement already satisfied: asn1crypto>=0.21.0 in /usr/local/lib/python3.6/site-packages (from cryptography>=1.3.4->pyOpenSSL>=16.1.0->-r requirements.txt (line 2))
Requirement already satisfied: packaging in /usr/local/lib/python3.6/site-packages (from cryptography>=1.3.4->pyOpenSSL>=16.1.0->-r requirements.txt (line 2))
Requirement already satisfied: setuptools>=11.3 in /usr/local/lib/python3.6/site-packages (from cryptography>=1.3.4->pyOpenSSL>=16.1.0->-r requirements.txt (line 2))
Requirement already satisfied: idna>=2.1 in /usr/local/lib/python3.6/site-packages (from cryptography>=1.3.4->pyOpenSSL>=16.1.0->-r requirements.txt (line 2))
Requirement already satisfied: pycparser in /usr/local/lib/python3.6/site-packages (from cffi>=1.4.1->cryptography>=1.3.4->pyOpenSSL>=16.1.0->-r requirements.txt (line 2))
Requirement already satisfied: pyparsing in /usr/local/lib/python3.6/site-packages (from packaging->cryptography>=1.3.4->pyOpenSSL>=16.1.0->-r requirements.txt (line 2))
Installing collected packages: elasticsearch
  Found existing installation: elasticsearch 1.6.0
    Uninstalling elasticsearch-1.6.0:
      Successfully uninstalled elasticsearch-1.6.0
Successfully installed elasticsearch-5.3.0

The errors multiply :slight_smile:

WARNING:elasticsearch:GET https://xxxxxxxxxxx:443/_search [status:N/A request:0.016s]
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 345, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 844, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 326, in connect
    ssl_context=context)
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 324, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 401, in wrap_socket
    _context=self, _session=session)
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 808, in __init__
    self.do_handshake()
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1061, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 683, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)

During handling of the above exception, another exception occurred:



During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 114, in perform_request
    response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 630, in urlopen
    raise SSLError(e)
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test.py", line 39, in <module>
    es.search()
  File "/usr/local/lib/python3.6/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/elasticsearch/client/__init__.py", line 625, in search
    doc_type, '_search'), params=params, body=body)
  File "/usr/local/lib/python3.6/site-packages/elasticsearch/transport.py", line 318, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/usr/local/lib/python3.6/site-packages/elasticsearch/connection/http_urllib3.py", line 120, in perform_request
    raise SSLError('N/A', str(e), e)
elasticsearch.exceptions.SSLError: ConnectionError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)) caused by: SSLError([SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749))