About running Rally with error

[danielmitterdorfer]

Hi @leozkl1,

@leozkl1

What is the definition of 100.0th percentile latency, the value of "index-append" operation in the report is over 1s.

the 100.0th percentile is the maximum value that has been encountered. So this means that one bulk index operation took at most 1018.55ms or 1.019s.

@leozkl1

The indexing time is about 58mins, but the total test just takes 30mins, why does this happen?

The indexing time is not wall clock time but total time spent indexing. For example, if you have 4 indexing threads that each index for 10 minutes (in parallel), 10 minutes have elapsed but the 4 indexing threads have spent 4 * 10 minutes = 40 minutes total. Makes sense?

@leozkl1

I cannot find disk io key in neither the report nor elasticsearch, this is different from the reference docs.

The metrics key is called disk_io_write_bytes (see docs). You can query the last 10 values with the following query:

GET /rally-*/metrics/_search
{
   "query": {
      "term": {
         "name": {
            "value": "disk_io_write_bytes"
         }
      }
   },
   "size": 10,
   "sort": [
      {
         "trial-timestamp": {
            "order": "desc"
         }
      }
   ]
}

On my machine a hit looks like this for example:

{
            "_index": "rally-2016",
            "_type": "metrics",
            "_id": "AVfXlVqoHtkxNiBSdqV6",
            "_score": null,
            "_source": {
               "@timestamp": 1476790599815,
               "environment": "local",
               "name": "disk_io_write_bytes",
               "relative-time": 3272043053,
               "trial-timestamp": "20161018T104207Z",
               "meta": {
                  "host_name": "taz",
                  "os_name": "Linux",
                  "node_name": "rally-node0",
                  "jvm_version": "1.8.0_102",
                  "cpu_logical_cores": 8,
                  "source_revision": "13e62e1",
                  "cpu_physical_cores": 4,
                  "cpu_model": "Intel(R) Xeon(R) CPU E3-1270 v5 @ 3.60GHz",
                  "jvm_vendor": "Oracle Corporation",
                  "os_version": "4.7.4-1-ARCH"
               },
               "value": 347668783104,
               "sample-type": "normal",
               "challenge": "append-no-conflicts",
               "car": "defaults",
               "track": "nyc_taxis",
               "unit": "byte"
            },
            "sort": [
               1476787327000
            ]
         }

@leozkl1

Does detailed definition of metrcs or operations described in the reference docs?

Metrics are defined in the reference docs. For operations you should look directly into the track repository (look in track.json of the track you're running) but you can choose any name for an operation. Are you missing something?

@leozkl1

By the way, searchguard is a opensource security plugin for elasticsearch, just like what shield does.

Didn't know that. Thanks for the info. I guess your problem then was that Rally did not authenticate at the cluster. You can use --client-options to pass username and password (see the examples in the docs; it's basically: --client-options="basic_auth_user:'user',basic_auth_password:'password',timeout:60000,request_timeout:60000"assuming that you don't use SSL, otherwise: please look in the docs).

Daniel