# Esrally Urllib3HttpConnection **URL:** \ **Category:** Elasticsearch\ **Tags:** rally\ **Created:** [April 6, 2020, 1:08am UTC](https://discuss.elastic.co/t/esrally-urllib3httpconnection/226601 "2020-04-06T01:08:05Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![kinlee](https://avatars.discourse-cdn.com/v4/letter/k/838e76/32.png) [@kinlee](https://discuss.elastic.co/u/kinlee)\ **Post date:** [April 6, 2020, 1:08am UTC](https://discuss.elastic.co/t/esrally-urllib3httpconnection/226601/1 "2020-04-06T01:08:06Z")
2020-04-06 00:59:42,445 -not-actor-/PID:15373 elasticsearch WARNING Connection \ has failed for 2 times in a row, putting on 120 second timeout. 2020-04-06 00:59:42,447 -not-actor-/PID:15375 elasticsearch WARNING POST [http://xxx.xxx.xxx.xxx9200/\_bulk](http://xxx.xxx.xxx.xxx9200/_bulk) [status:503 request:0.091s] 2020-04-06 00:59:42,466 -not-actor-/PID:15373 elasticsearch WARNING Undecodable raw error response from server: Expecting value: line 1 column 1 (char 0) I get this in my rallylog, I don't know why. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 6, 2020, 7:30am UTC](https://discuss.elastic.co/t/esrally-urllib3httpconnection/226601/2 "2020-04-06T07:30:08Z")
Can you curl the Elasticsearch node from the rally host? ---
**Author:** ![kinlee](https://avatars.discourse-cdn.com/v4/letter/k/838e76/32.png) [@kinlee](https://discuss.elastic.co/u/kinlee)\ **Post date:** [April 6, 2020, 7:42am UTC](https://discuss.elastic.co/t/esrally-urllib3httpconnection/226601/3 "2020-04-06T07:42:21Z")
yes i can curl from rally host.... I dont if i am making a mistake in using the command esrally --distribution-version=7.4.2 --track pmc --target-hosts=x.x.x.x9200,x.x.x.x:9200,x.x.x.x:9200 --pipeline=benchmark-only --client-options="use\_ssl:true,basic\_auth\_user:'username',basic\_auth\_password:'password',verify\_certs:false" ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 6, 2020, 8:59am UTC](https://discuss.elastic.co/t/esrally-urllib3httpconnection/226601/4 "2020-04-06T08:59:16Z")
> [@kinlee](#): > > \ has failed for 2 times in a row, putting on 120 second timeout. It seems like Rally tries to connect through HTTP even though it looks like you have a secured cluster. Can you try and change the connection string to include `https://` and see if that makes any difference? ---
**Author:** ![dliappis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dliappis/32/56174_2.png) [@dliappis](https://discuss.elastic.co/u/dliappis)\ **Post date:** [April 6, 2020, 9:19am UTC](https://discuss.elastic.co/t/esrally-urllib3httpconnection/226601/5 "2020-04-06T09:19:42Z")
> [@kinlee](#): > > 2020-04-06 00:59:42,447 -not-actor-/PID:15375 elasticsearch WARNING POST [http://xxx.xxx.xxx.xxx9200/\_bulk](http://xxx.xxx.xxx.xxx9200/_bulk) [status:503 request:0.091s] Since Elasticsearch is reporting `503` error, have you also checked in the Elasticsearch logs for clues? ---
**Author:** ![kinlee](https://avatars.discourse-cdn.com/v4/letter/k/838e76/32.png) [@kinlee](https://discuss.elastic.co/u/kinlee)\ **Post date:** [April 6, 2020, 10:49am UTC](https://discuss.elastic.co/t/esrally-urllib3httpconnection/226601/6 "2020-04-06T10:49:09Z")
I can see pmc index in my cluster, many time rally will fail at this point .track.params INFO Will read [574199] lines from [/home/centos/.rally/benchmarks/data/pmc/documents.json] starting from line [0] with bulk size [1]. surprisingly, esrally will continue and will give some result but in the rally log i get timeout error. in the elastic search i get ELASTICSEARCH.TRANSPORT.RemoteTransportException. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 6, 2020, 10:54am UTC](https://discuss.elastic.co/t/esrally-urllib3httpconnection/226601/7 "2020-04-06T10:54:57Z")
Please show the full error from the Elasticsearch logs. ---
**Author:** ![kinlee](https://avatars.discourse-cdn.com/v4/letter/k/838e76/32.png) [@kinlee](https://discuss.elastic.co/u/kinlee)\ **Post date:** [April 6, 2020, 11:08am UTC](https://discuss.elastic.co/t/esrally-urllib3httpconnection/226601/8 "2020-04-06T11:08:27Z")
Hi there is not much in elasticsearch log but rally is showing elasticsearch WARNING POST [https://34.192.225.69:9200/\_bulk](https://34.192.225.69:9200/_bulk) [status:N/A request:10.010s] Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 421, in \_make\_request six.raise\_from(e, None) File "", line 3, in raise\_from File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 416, in \_make\_request httplib\_response = conn.getresponse() File "/usr/lib64/python3.6/http/client.py", line 1346, in getresponse response.begin() File "/usr/lib64/python3.6/http/client.py", line 307, in begin version, status, reason = self.\_read\_status() File "/usr/lib64/python3.6/http/client.py", line 268, in \_read\_status line = str(self.fp.readline(\_MAXLINE + 1), "iso-8859-1") File "/usr/lib64/python3.6/socket.py", line 586, in readinto return self.\_sock.recv\_into(b) File "/usr/lib64/python3.6/ssl.py", line 968, in recv\_into return self.read(nbytes, buffer) File "/usr/lib64/python3.6/ssl.py", line 830, in read return self.\_sslobj.read(len, buffer) File "/usr/lib64/python3.6/ssl.py", line 587, in read v = self.\_sslobj.read(len, buffer) socket.timeout: The read operation timed out During handling of the above exception, another exception occurred: ---
**Author:** ![dliappis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dliappis/32/56174_2.png) [@dliappis](https://discuss.elastic.co/u/dliappis)\ **Post date:** [April 6, 2020, 5:11pm UTC](https://discuss.elastic.co/t/esrally-urllib3httpconnection/226601/9 "2020-04-06T17:11:19Z")
The `pmc` track has [very large docs](https://github.com/elastic/rally-tracks/tree/master/pmc#example-document) and can cause stability issues to your cluster if it's underprovisioned, especially in terms of memory. As suggested by @Christian_Dahlqvist there should be some useful entries in the Elasticsearch log, but you can also check a) if regular curl operations against ES work when Rally reports time outs b) memory/cpu/disk/network usage on the Elasticsearch and Rally nodes. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 4, 2020, 5:11pm UTC](https://discuss.elastic.co/t/esrally-urllib3httpconnection/226601/10 "2020-05-04T17:11:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.