Trouble configuring datastore

[lzukel (Lance)]

I am unable to use an existing elasticsearch cluster as a datastore. I am unable to find how to disable certificate verification for the datastore, thus the only way I can run rally is to use the in-memory datastore.

I can set verify_ssl False for the cluster that the tests are being run against. But unable to set the value for the datastore.

[lzukel (Lance)]

Error from Rally:

[ERROR] Cannot race. Error in race control (('Could not connect to your Elasticsearch metrics store. Please check that it is running on host [server0080] at port [9200] or fix the configuration in [/root/.rally/rally.ini].', None))

From rally logs:

2020-07-22 14:26:44,844 ActorAddr-(T|:38297)/PID:14908 esrally.client INFO Creating ES client connected to [{'host': 'server0080', 'port': '9200'}] with options [{'use_ssl': True, 'verify_certs': True, 'timeout': 120, 'basic_auth_user': 'admin', 'basic_auth_password': '*****'}]
2020-07-22 14:26:44,844 ActorAddr-(T|:38297)/PID:14908 esrally.client INFO SSL support: on
2020-07-22 14:26:44,850 ActorAddr-(T|:38297)/PID:14908 esrally.client INFO SSL certificate verification: on
2020-07-22 14:26:44,850 ActorAddr-(T|:38297)/PID:14908 esrally.client INFO SSL client authentication: off
2020-07-22 14:26:44,850 ActorAddr-(T|:38297)/PID:14908 esrally.client INFO HTTP basic authentication: on
2020-07-22 14:26:44,850 ActorAddr-(T|:38297)/PID:14908 esrally.client INFO HTTP compression: off
2020-07-22 14:26:44,894 ActorAddr-(T|:38297)/PID:14908 esrally.metrics ERROR Could not connect to your Elasticsearch metrics store. Please check that it is running on host [server0080] at port [9200] or fix the configuration in [/root/.rally/rally.ini].
...
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:877)

Rally.ini:

[meta]
config.version = 17

[system]
env.name = gis

[node]
root.dir = /root/.rally/benchmarks
src.root.dir = /root/.rally/benchmarks/src

[source]
remote.repo.url = https://github.com/elastic/elasticsearch.git
elasticsearch.src.subdir = elasticsearch

[benchmarks]
local.dataset.cache = /root/.rally/benchmarks/data

[reporting]
#datastore.type = in-memory
datastore.type = elasticsearch
datastore.host = server0080
datastore.port = 9200
datastore.secure = True
datastore.user = admin
datastore.password = *****

[tracks]
default.url = https://github.com/elastic/rally-tracks

[teams]
default.url = https://github.com/elastic/rally-teams

[defaults]
preserve_benchmark_candidate = False

[distributions]
release.cache = true

Rally run command:

esrally --track-path=/root/.rally/benchmarks/tracks/default --offline --pipeline=benchmark-only --target-hosts=elasticsearch:443/es_search --client-options="use_ssl:true,verify_certs:false,basic_auth_user:'admin',basic_auth_password:'******'"

This configuration works when datastore is in-memory... But unable to store results in elasticsearch cluster, due to unable to disable verify_certs for the datastore...

[baz (Michael Basnight)]

Hi,

you are looking for datastore.ssl.verification_mode which should be set to none in the case that you do not want to verify_certs. See the docs here for more information.

[lzukel (Lance)]

Thank you this solved my issue

[baz (Michael Basnight)]

excellent!!