ESRally version 2.7.0 is throwing version mismatch h error with elasticsearch server 7.6.1

Elastic Stack Elasticsearch
1

elasticsearch.exceptions.UnsupportedProductError: The client noticed that the server is not a supported distribution of Elasticsearch

I have installed esrally 2.7.0 on my local mac machine in a separate python virtual env and trying to run sample race by connecting to sso protected elasticsearch cluster, but getting below error:

Command:
esrally race --track=geopoint --target-hosts=es-dev-1-:9200,es-dev-2-:9200,es-dev-3-*:9200 --pipeline=benchmark-only --client-options="use_ssl:true,verify_certs:false,basic_auth_user:,basic_auth_password:"

Error:
elasticsearch.exceptions.UnsupportedProductError: The client noticed that the server is not a supported distribution of Elasticsearch

Please help me here. Thanks in advance.

2

Hi @Rahul_Prajapati - thanks for your interest in Rally.

This error is returned because Rally's elasticsearch-py client couldn't verify that the target cluster is actually an Elasticsearch cluster (see Verify connection to Elasticsearch by sethmlarson · Pull Request #1623 · elastic/elasticsearch-py · GitHub).

I tried this locally and didn't have any issues benchmarking 7.6.1:

$ esrally --version
esrally 2.7.0

Cluster:

$ curl -k -u elastic:changeme https://localhost:9200
{
  "name" : "elasticsearch1",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "s09wHgUlS26MyBi9fDk9XQ",
  "version" : {
    "number" : "7.6.1",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "aa751e09be0a5072e8570670309b1f12348f023b",
    "build_date" : "2020-02-29T00:15:25.529771Z",
    "build_snapshot" : false,
    "lucene_version" : "8.4.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

$ curl -k -u elastic:changeme https://localhost:9200/_license
{
  "license" : {
    "status" : "active",
    "uid" : "07ae4064-3b4a-4dd4-b0c8-d7832c453281",
    "type" : "basic",
    "issue_date" : "2023-02-05T23:35:55.098Z",
    "issue_date_in_millis" : 1675640155098,
    "max_nodes" : 1000,
    "issued_to" : "docker-cluster",
    "issuer" : "elasticsearch",
    "start_date_in_millis" : -1
  }
}

Invocation:

$ esrally race --track=geopoint  --target-hosts=localhost:9200 --pipeline=benchmark-only --client-options="use_ssl:true,verify_certs:false,basic_auth_user:elastic,basic_auth_password:changeme" --test-mode

    ____        ____
   / __ \____ _/ / /_  __
  / /_/ / __ `/ / / / / /
 / _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
                /____/

[INFO] Race id is [b8be0843-02cd-4ab8-988a-a83932d82db1]
[INFO] Racing on track [geopoint], challenge [append-no-conflicts] and car ['external'] with version [7.6.1].

[WARNING] merges_total_time is 569927 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] indexing_total_time is 3297659 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
[WARNING] refresh_total_time is 349035 ms indicating that the cluster is not in a defined clean state. Recorded index time metrics may be misleading.
Running delete-index                                                           [100% done]
Running create-index                                                           [100% done]
Running check-cluster-health                                                   [100% done]
Running index-append                                                           [100% done]
Running refresh-after-index                                                    [100% done]
[...]
--------------------------------
[INFO] SUCCESS (took 32 seconds)
--------------------------------

That said, 7.6.1 has been End of Life for quite some time now, have you considered upgrading?

3

Hi @Bradley_Deam
Thanks for your quick response. Actually I am also able to connect to locally installed ES successfully. race for geopoit was success locally. Then I proceeded for remote ES where actually I want to perform load testing.

If I am providing correct ES host and credentials then I am not getting why Rally's elasticsearch-py client couldn't verify that the target cluster is actually an Elasticsearch cluster. Please suggest me if my command is not correct.

esrally race --track=geopoint --target-hosts=:9200,:9200,:9200 --pipeline=benchmark-only --client-options="use_ssl:true,verify_certs:false,basic_auth_user:,basic_auth_password:"

4

There shouldn't be any difference between Rally connecting to a local cluster vs a remote cluster in terms of verification.

Can you tell me more about this remote ES? Is this an AWS Managed Elasticsearch deployment?