transport = Transport([{'host': '<your_host1>'}, {'host': '<your_host2>'}], http_auth=('<your_user>', '<user_password>'))
print('no of connections in connection pool before adding connection = ' + str(len(transport.connection_pool.connections)))
#you may increase the range value from 2 to required no. as per your requirement
for cntr in range(2):
transport.add_connection({'host': '<your_host1>'})
transport.add_connection({'host': '<your_host2>'})
print('no of connections in connection pool after adding connection = ' + str(len(transport.connection_pool.connections)))
cnn1 = transport.get_connection()
cnn2 = transport.get_connection()
cnn3 = transport.get_connection()
cnn4 = transport.get_connection()
cnn5 = transport.get_connection()
cnn6 = transport.get_connection()
cnn7 = transport.get_connection()
cnn8 = transport.get_connection()
cnn9 = transport.get_connection()
cnn10 = transport.get_connection()
result = transport.perform_request(method='GET', url='/<index_name_to_query>/_search', body=qry)
for doc in result['hits']['hits']:
print('got data...')
except Exception as e:
print('exception...')
print(str(e))
Hi bry-c,
Thanks for the link.
I was initially using the Elasticsearch class as below:
from elasticsearch import Elasticsearch
....
elastic_conn = Elasticsearch(['<my_host>'], http_auth=('<my_user>', '<my_user_pwd>'))
qry = {"query": {"bool": {"must": [{"match": {"extension": "css"}}, {"match": {"machine.os": "ios"}}]}}}
res = elastic_conn.search(index="<my_index_name>", body=qry)
for doc in res['hits']['hits']:
print('got data...')
But, I need to implement connection pooling at server start up & use a connection from this pool (by using --> transport.get_connection()) for all future queries. I could not find a link to implement connection pooling using the above, hence did connection pooling using the "Transport" class.
How can I implement connection pooling (at server start up) as per your link?
bry-c,
Thanks for your comments.
I am using flask & apache2. My existing connection pooling implementation is similar to what you have mentioned.
After your reply, I re-read the following links/sections which gave me the required details & clarifications I had missed out initially
#maxsize parameter for connection poolsize
es = Elasticsearch(["host1", "host2"], maxsize=25)
Query
Here, I am assuming that the parameter "maxsize" is for no. of persistent connections per confirgured node.
As per the thread-safety link above :
If your application is long-running consider turning on Sniffing to make sure the client is up to date on the cluster location.
My application is long running almost 24x7, so should I turn on sniffing mechanism?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.