# Trying to load data using Rally into Elastic Search **URL:** \ **Category:** Elasticsearch\ **Tags:** rally\ **Created:** [April 12, 2018, 8:27pm UTC](https://discuss.elastic.co/t/trying-to-load-data-using-rally-into-elastic-search/127875 "2018-04-12T20:27:36Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![sarba](https://avatars.discourse-cdn.com/v4/letter/s/dc4da7/32.png) [@sarba](https://discuss.elastic.co/u/sarba)\ **Post date:** [April 12, 2018, 8:27pm UTC](https://discuss.elastic.co/t/trying-to-load-data-using-rally-into-elastic-search/127875/1 "2018-04-12T20:27:36Z")
Trying to load data using rally into Elastic Search and facing this issue: root@esdb01-cqs01:~/.rally/benchmarks/tracks/default/contactdata# esrally --track=contactdata --target-hosts=[esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com:9200](http://esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com:9200) --track-params="bulk\_size:2000" --pipeline=benchmark-only --client-options="use\_ssl:true,verify\_certs:true,basic\_auth\_user:'elastic',basic\_auth\_password:''" ``` ________ ``` / \_\_ \_\_\_\_ _/ / /_ \_\_ / /_/ / \_\_ `/ / / / / / / , / // / / / // / /_/ |_|\ __,_/_/_/\__ , / /\_\_\_\_/ [INFO] You did not provide an explicit timeout in the client options. Assuming default of 10 seconds. [INFO] Writing logs to /root/.rally/logs/rally\_out\_20180412T201810Z.log * * * \*\*\*\*\*\*\*\*\*\*\*\*\*\* WARNING: A dark dungeon lies ahead of you \*\*\*\*\*\*\*\*\*\*\*\*\*\* * * * Rally does not have control over the configuration of the benchmarked Elasticsearch cluster. Be aware that results may be misleading due to problems with the setup. Rally is also not able to gather lots of metrics at all (like CPU usage of the benchmarked cluster) or may even produce misleading metrics (like the index size). * * * \*\*\*\*\*\* Use this pipeline only if you are aware of the tradeoffs. \*\*\*\*\*\* \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* Watch your step! \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* * * * [WARNING] Could not terminate all internal processes within timeout. Please check and force-terminate all Rally processes. [ERROR] Cannot race. Got an unexpected result during benchmarking: [Poison\<\\>: Traceback (most recent call last): File "/usr/local/lib/python3.4/dist-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/usr/local/lib/python3.4/dist-packages/urllib3/connectionpool.py", line 346, in \_make\_request self.\_validate\_conn(conn) File "/usr/local/lib/python3.4/dist-packages/urllib3/connectionpool.py", line 850, in _validate\_conn conn.connect() File "/usr/local/lib/python3.4/dist-packages/urllib3/connection.py", line 326, in connect ssl\_context=context) File "/usr/local/lib/python3.4/dist-packages/urllib3/util/ssl_.py", line 329, in ssl\_wrap\_socket return context.wrap\_socket(sock, server\_hostname=server\_hostname) File "/usr/lib/python3.4/ssl.py", line 365, in wrap\_socket \_context=self) File "/usr/lib/python3.4/ssl.py", line 601, in **init** self.do\_handshake() File "/usr/lib/python3.4/ssl.py", line 828, in do\_handshake self.\_sslobj.do\_handshake() ssl.SSLError: [SSL: UNKNOWN\_PROTOCOL] unknown protocol (\_ssl.c:600) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.4/dist-packages/elasticsearch/connection/http\_urllib3.py", line 147, in perform\_request response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, \*\*kw) File "/usr/local/lib/python3.4/dist-packages/esrally/client.py", line 47, in urlopen return self.pool.urlopen(method, url, body=body, retries=retries, headers=headers, \*\*kw) File "/usr/local/lib/python3.4/dist-packages/urllib3/connectionpool.py", line 639, in urlopen \_stacktrace=sys.exc\_info()[2]) File "/usr/local/lib/python3.4/dist-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, \_stacktrace) File "/usr/local/lib/python3.4/dist-packages/urllib3/packages/six.py", line 685, in reraise raise value.with\_traceback(tb) File "/usr/local/lib/python3.4/dist-packages/urllib3/connectionpool.py", line 601, in urlopen chunked=chunked) File "/usr/local/lib/python3.4/dist-packages/urllib3/connectionpool.py", line 346, in \_make\_request self.\_validate\_conn(conn) File "/usr/local/lib/python3.4/dist-packages/urllib3/connectionpool.py", line 850, in _validate\_conn conn.connect() File "/usr/local/lib/python3.4/dist-packages/urllib3/connection.py", line 326, in connect ssl\_context=context) File "/usr/local/lib/python3.4/dist-packages/urllib3/util/ssl_.py", line 329, in ssl\_wrap\_socket return context.wrap\_socket(sock, server\_hostname=server\_hostname) File "/usr/lib/python3.4/ssl.py", line 365, in wrap\_socket \_context=self) File "/usr/lib/python3.4/ssl.py", line 601, in **init** self.do\_handshake() File "/usr/lib/python3.4/ssl.py", line 828, in do\_handshake self.\_sslobj.do\_handshake() urllib3.exceptions.SSLError: [SSL: UNKNOWN\_PROTOCOL] unknown protocol (\_ssl.c:600) ---
**Author:** ![sarba](https://avatars.discourse-cdn.com/v4/letter/s/dc4da7/32.png) [@sarba](https://discuss.elastic.co/u/sarba)\ **Post date:** [April 12, 2018, 8:47pm UTC](https://discuss.elastic.co/t/trying-to-load-data-using-rally-into-elastic-search/127875/2 "2018-04-12T20:47:02Z")
I am using Elastic Search 6.2.2 with X-pack ---
**Author:** ![dliappis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dliappis/32/56174_2.png) [@dliappis](https://discuss.elastic.co/u/dliappis)\ **Post date:** [April 13, 2018, 7:27am UTC](https://discuss.elastic.co/t/trying-to-load-data-using-rally-into-elastic-search/127875/3 "2018-04-13T07:27:50Z")
Hello @sarba, The client options you are using instruct Rally to use `https` (SSL/TLS): ```auto --client-options="use_ssl:true,verify_certs:true ``` however, the error message you are receiving indicates that Rally failed trying to use the encrypted http protocol with TLS/SSL to talk to Elasticsearch at the provided port. One possible reason is that Elasticsearch hasn't been properly configured to use [SSL for the http client](https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls.html#tls-http). This is easy to check using something like: `curl https://esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com:9200` which will return an error if it's unable to establish the encrypted connection, such as: ```auto curl: (35) gnutls_handshake() failed: An unexpected TLS packet was received. ``` (one alternative method of checking whether Elasticsearch is accepting TLS encrypted http communication is with `openssl s_client -showcerts -connect esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com:9200`; it will exit showing "no peer certificate available" if SSL/TLS isn't supported). ---
**Author:** ![sarba](https://avatars.discourse-cdn.com/v4/letter/s/dc4da7/32.png) [@sarba](https://discuss.elastic.co/u/sarba)\ **Post date:** [April 13, 2018, 2:51pm UTC](https://discuss.elastic.co/t/trying-to-load-data-using-rally-into-elastic-search/127875/4 "2018-04-13T14:51:10Z")
Thanks for your reply. root@esdb01-cqs01:~/.rally/logs# curl -u elastic:'' -XGET '[http://esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com:9200](http://esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com:9200)' { "name" : "[esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com](http://esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com)", "cluster\_name" : "elastic.cqs01.stg.stage", "cluster\_uuid" : "zOvq3bVkQfeCB\_OHAf9vMQ", "version" : { "number" : "6.2.2", "build\_hash" : "10b1edd", "build\_date" : "2018-02-16T19:01:30.685723Z", "build\_snapshot" : false, "lucene\_version" : "7.2.1", "minimum\_wire\_compatibility\_version" : "5.6.0", "minimum\_index\_compatibility\_version" : "5.0.0" }, "tagline" : "You Know, for Search" } root@esdb01-cqs01:~/.rally/logs# curl -u elastic:'' -XGET '[https://esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com:9200](https://esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com:9200)' curl: (35) error:140770FC:SSL routines:SSL23\_GET\_SERVER\_HELLO:unknown protocol root@esdb01-cqs01:~/.rally/logs# ## root@esdb01-cqs01:~/.rally/logs# openssl s\_client -showcerts -connect [esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com:9200](http://esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com:9200) CONNECTED(00000003) 139774469691040:error:140770FC:SSL routines:SSL23\_GET\_SERVER\_HELLO:unknown protocol:s23\_clnt.c:795: ## no peer certificate available ## No client certificate CA names sent ## SSL handshake has read 7 bytes and written 295 bytes ## New, (NONE), Cipher is (NONE) Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE root@esdb01-cqs01:~/.rally/logs# But Elastic search nodes are using secure connection and I have certificates and x-pack is installed. ---
**Author:** ![sarba](https://avatars.discourse-cdn.com/v4/letter/s/dc4da7/32.png) [@sarba](https://discuss.elastic.co/u/sarba)\ **Post date:** [April 13, 2018, 3:47pm UTC](https://discuss.elastic.co/t/trying-to-load-data-using-rally-into-elastic-search/127875/5 "2018-04-13T15:47:06Z")
I have enabled Https. Now it shows this error: root@esdb01-cqs01:/etc/elasticsearch# curl [https://esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com:9200](https://esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com:9200) curl: (60) SSL certificate problem: self signed certificate in certificate chain More details here: [http://curl.haxx.se/docs/sslcerts.html](http://curl.haxx.se/docs/sslcerts.html) curl performs SSL certificate verification by default, using a "bundle" of Certificate Authority (CA) public keys (CA certs). If the default bundle file isn't adequate, you can specify an alternate file using the --cacert option. If this HTTPS server uses a certificate signed by a CA represented in the bundle, the certificate verification probably failed due to a problem with the certificate (it might be expired, or the name might not match the domain name in the URL). If you'd like to turn off curl's verification of the certificate, use the -k (or --insecure) option. curl -u elastic:'' [https://esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com:9200](https://esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com:9200) -k { "name" : "[esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com](http://esdb01-cqs01.db.us-west-1a.stg1.ebs.ebcolo.com)", "cluster\_name" : "elastic.cqs01.stg.stage", "cluster\_uuid" : "zOvq3bVkQfeCB\_OHAf9vMQ", "version" : { "number" : "6.2.2", "build\_hash" : "10b1edd", "build\_date" : "2018-02-16T19:01:30.685723Z", "build\_snapshot" : false, "lucene\_version" : "7.2.1", "minimum\_wire\_compatibility\_version" : "5.6.0", "minimum\_index\_compatibility\_version" : "5.0.0" }, "tagline" : "You Know, for Search" } ---
**Author:** ![dliappis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dliappis/32/56174_2.png) [@dliappis](https://discuss.elastic.co/u/dliappis)\ **Post date:** [April 14, 2018, 12:58pm UTC](https://discuss.elastic.co/t/trying-to-load-data-using-rally-into-elastic-search/127875/6 "2018-04-14T12:58:42Z")
Hey @sarba, Now that you've properly enabled SSL (with self signed certificates, it appears) you should be able to execute Rally with a minor change in your arguments: `verify_certs:true` --\> `verify_certs:false` ---
**Author:** ![sarba](https://avatars.discourse-cdn.com/v4/letter/s/dc4da7/32.png) [@sarba](https://discuss.elastic.co/u/sarba)\ **Post date:** [April 16, 2018, 5:41pm UTC](https://discuss.elastic.co/t/trying-to-load-data-using-rally-into-elastic-search/127875/7 "2018-04-16T17:41:32Z")
Yes, it is working when I specify verify\_certs:false, but failed when specify : verify\_certs:true. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 14, 2018, 5:41pm UTC](https://discuss.elastic.co/t/trying-to-load-data-using-rally-into-elastic-search/127875/8 "2018-05-14T17:41:40Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.