Just wondering if your test is correct. I mean: what do you want to test? If ES can deal with 1000 search requests?
If it's your question, you should parallelize your tests. In java, you should create more test Threads. I don't know with python and pyes.
It seems that with your test, you create 1000 calls, one by one. Just like if you were creating 1000 curl http://www.google.com and see how long it takes...
My 2 cents.
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
Le 23 août 2012 à 23:25, Abhishek Pratap email@example.com a écrit :
Guys I am stuck and need some guidance in order to move fwd with ES and use it.
Mainly the bottle neck is #search queries I am able to make ( 1000 queries in 2-3 seconds). Can this be scaled up ?
I have also asked this on stackoverflow but dint get any response.
On Wednesday, August 22, 2012 11:13:58 AM UTC-7, Abhishek Pratap wrote:
And just in case anyone is interested this is how I am testing the search performance
loop_start = time.clock()
q1 = TermQuery("tax_name","cellvibrio")
for x in xrange(1000000):
if x % 1000 == 0 and x > 0:
loop_check_point = time.clock()
print 'took %s secs to search %d records' % (loop_check_point-loop_start,x)
results = conn.search(query=q1)
for r in results:
On Wednesday, August 22, 2012 10:27:53 AM UTC-7, Abhishek Pratap wrote:
Sorry I will have to push this again. I am still not able to get an optimum performance from ES for searches.
my index contains 1.5 million records and I am able to make 800-1000 searches in 2 seconds using pyes. It has been a while since we are trying to optimize ES through ES for our production work. Any help now will be appreciated.
On Monday, August 20, 2012 9:48:31 AM UTC-7, Abhishek Pratap wrote:
I have upgraded the requests module. Request version 0.13.8 and pyes version 0.19..
Still able to make only about 1000 searches per 2-3 second period
On Thursday, August 16, 2012 2:43:28 PM UTC-7, Anton2 wrote:
Be sure to use the latest version of pyes and requests, or the development version of pyes from github (which goes back to use urllib3).
We had troubles with certain combination of pyes and requests a few month ago. Requests was fetching data from the network one byte at a time, with abysmal performances…
Apologies for pushing it once more. Can any one help me figure out why my search queries on ES are slow. I am able to get 1000 results in 2-3 seconds(details in the first post of this thread). I would expect the performance to be atleast 50-100 times more faster than this. I hope thats a realistic expectation.