Python elasticsearch performance

[revelc33 (Martin M)]

i am trying to make 300 queries per second to my Elasticsearch from my script, but i am not very sure how can i make in an optimal way reusing the connections.

`
from elasticsearch import Elasticsearch
import _thread

es = Elasticsearch( [ { 'host' : "127.0.0.1" 'port': 9200 , 'max_clients' : 100000 } ] )

def myThreadFunc(es):
result = es.msearch( body = myquery, max_concurrent_searches = 100 ,request_timeout = 0.6)
makeSomethingwithResult(result)
return 0

whille True:
for i in range(10):
_thread.start_new_thread(myThreadFunc,(es),)
time.sleep(1)
`
is the "es" object going to deal with reusing connections and keep alives( if it is necessary)?

When i check GET /_nodes/stats on my ES the 'http' -> "total_opened" the number increase very quickly. If the connections are keeped alive it's supposed this value has to increase only when connections are expired and opened again?

[revelc33 (Martin M)]

@honzakral @Honza_Kral

[honzakral (Honza Král)]

Hi! Yes, the es object is thread safe and will indeed reuse the connection. There can still be an increase in the connections (not to mention that every time you check it it will increase as well) but that should stabilize over time.