Running Rally w/ use_ssl — how do I pass cert & key info?

[khyron4eva]

Hello again.

After having finally located the docs re: --client-options, I've been trying some invocations with "use_ssl:true" but they all fail. Rally won't let me disable cert verification so now I'm stuck. I could use some guidance on how to pass both cert and auth information to Rally.

When I use curl(1) commands against these clusters, I normally have to specify the "-k", "-u", "--cert" and "--key" flags. How can I pass this information to Rally to pass to the clusters? The ES node (single node cluster) has a cert and key for 2-way SSL but I’m running Rally on a remote node which does not have access to that cert/key pair.

Environment is Rally 0.5.3.dev0 on CentOS 6.9 (client & server). ES 1.4.4. Single node cluster. My current Rally command is:

esrally --distribution-version=1.4.4 --pipeline=benchmark-only --target-hosts=:9200 --client-options="use_ssl:true"

When I copy the cert file from the ES server to the Rally client and specify the command as follows:

--client-options="use_ssl:true,ca_cert:///cert.cer"

I receive the following error/traceback:

.
.
Traceback (most recent call last):
File "/usr/bin/esrally", line 9, in
load_entry_point('esrally==0.5.3.dev0', 'console_scripts', 'esrally')()
File "/usr/lib/python3.4/site-packages/esrally-0.5.3.dev0-py3.4.egg/esrally/rally.py", line 557, in main
client_options = kv_to_map(csv_to_list(args.client_options))
File "/usr/lib/python3.4/site-packages/esrally-0.5.3.dev0-py3.4.egg/esrally/rally.py", line 477, in kv_to_map
result[k.strip()] = convert(v.strip())
File "/usr/lib/python3.4/site-packages/esrally-0.5.3.dev0-py3.4.egg/esrally/rally.py", line 471, in convert
return to_bool(v)
File "/usr/lib/python3.4/site-packages/esrally-0.5.3.dev0-py3.4.egg/esrally/rally.py", line 449, in to_bool
raise ValueError("Could not convert value '%s'" % v)
ValueError: Could not convert value ‘///cert.cer'

I figure I have to be missing something simple but I haven't found it in the docs yet. Any guidance is most appreciated.

Cheers!

[danielmitterdorfer]

Hi @khyron4eva,

a couple of things are going wrong here (and I'd blame the docs for this!):

1. You need to enclose string values in single quotes (that's the reason why it complains that it cannot convert the value to a boolean)
2. the key is called ca_certs (not the "s" at the end)

So I think this should work: --client-options="use_ssl:true,ca_certs:'/cert.cer'" (provided your certificate is located at/cert.cer`). It would be great if you could report back if that worked. Then I'll update the docs.

A workaround is to disable certificate verification with verify_certs:false (the full options are --client-options="use_ssl:true,verify_certs:false" in this case).

Daniel

[khyron4eva]

@danielmitterdorfer,

Failed. See below:

[claudej@ ~]$ esrally --distribution-version=1.4.4 --pipeline=benchmark-only --target-hosts=:9200 --client-options="use_ssl:true,verify_certs:true,ca_certs:'/home/claudej/rally/cert.cer'"
.
.
[ERROR] Cannot race. ConnectionError(EOF occurred in violation of protocol (_ssl.c:600)) caused by: SSLError(EOF occurred in violation of protocol (_ssl.c:600))
.
.

And as I mentioned, using "verify_certs:false" tells me it is insecure and won't work:

usr/lib/python3.4/site-packages/elasticsearch-5.1.0-py3.4.egg/elasticsearch/connection/http_urllib3.py:89: UserWarning: Connecting to using SSL with verify_certs=False is insecure.
[ERROR] Cannot race. ConnectionError(EOF occurred in violation of protocol (_ssl.c:600)) caused by: SSLError(EOF occurred in violation of protocol (_ssl.c:600))

Cheers!

[khyron4eva]

Replying to myself but could this be due to not specifying the key with the cert file? Is there a syntax to do so? I imagine the CA certs are hardcoded into Rally and so I'm not trying to specify those, just the cert & key that clients need to use to talk to ES.

[danielmitterdorfer]

Hi @khyron4eva,

thanks for checking. I think this is a real issue then and needs to be fixed. Would you mind opening an issue on Github? Thanks!

Daniel

[khyron4eva]

Will do...tomorrow!

Cheers!

[khyron4eva]

@danielmitterdorfer Opened https://github.com/elastic/rally/issues/277

Cheers!

[danielmitterdorfer]

Hi @khyron4eva,

thanks for opening. I'll see when I can get a fix out.

Daniel