Elasticsearch search performance

I use elasticsearch-py to search doc in elasticsearch . when i try to search doc under concurrent conditions , the time consumerd increased . I computed and traced the time consumed by urllib3(elasticsearch-py default http client) 。It seem like the same batch of requests took time to increase cumulatively。

the log format is :
<conneciton object,thread name, the request num summed up in the urllib3 pool , time consumed by http request > . Different threads saw the same "request num " can be recognized as the were in the same batch 。 Such as : the first connection that returns the result with "request num" 810 consume time 0.006xx ms , it was normal 。 However, the subsequent request consumed time is constantly accumulating。 I want to know if there is any synchronization on the server side.

elasticsearh version is 6.4.2, elasticsearch-py version 6.3.1
elasticsearch search thread pool config is :
thread_pool.search.queue_size: 500
thread_pool.search.size: 100
thread_pool.search.min_queue_size: 10
thread_pool.search.max_queue_size: 1000
thread_pool.search.auto_queue_frame_size: 2000
thread_pool.search.target_response_time: 6s

query is simple:
{ 'query': {'match': {'question': 'xxxxx'}}, 'size': 10, 'from': 0}

In order to verify the problem, I did not use the any plugins .


1 Like

hi , is there any method use the elasticsearch-py with gevent ?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.