[ERROR] Cannot race. Error in track preparator (("'index' is mandatory", None))


(Emil Mirzayev) #1

Hi. When trying to create a custom race which involves 4 indices, i get the following error:

[ERROR] Cannot race. Error in track preparator (("'index' is mandatory", None))

I will share my track.json here:

{
  "name": "Sample track",
  "indices": [
    {
      "name": "devices",
      "auto-managed": false,
      "body": "devices-index.json",
      "types": [ "device" ]
    },
	{
      "name": "alarms",
      "auto-managed": false,
      "body": "alarm-index.json",
      "types": [ "alarm" ]
    },
	{
      "name": "measurements",
      "auto-managed": false,
      "body": "measurements-index.json",
      "types": [ "measurement" ]
    },
	{
      "name": "vee",
      "auto-managed": false,
      "body": "vee-index.json",
      "types": [ "vee_statistic" ]
    }
  ],
  "corpora": [
    {
      "name": "devices",
      "documents": [
        {
          "source-file": "/home/.../generated_devices.json",
          "document-count": 30,
		  "uncompressed-bytes": 15229,
          "target-type": "device",
          "target-index": "devices"
        }
      ]
    },
        {
      "name": "alarms",
      "documents": [
        {
          "source-file": "/home/.../generated_alarms.json",
          "document-count": 300,
		  "uncompressed-bytes": 1391,
          "target-type": "alarm",
          "target-index": "alarms"
        }
      ]
    },
	    {
      "name": "measurements",
      "documents": [
        {
          "source-file": "/home/.../generated_measurements.json",
          "document-count": 11520,
		  "uncompressed-bytes": 4693038,
          "target-type": "measurement",
          "target-index": "measurements"
        }
      ]
    },
	    {
      "name": "vee",
      "documents": [
        {
          "source-file": "/home/.../generated_vee.json",
          "document-count": 120,
		  "uncompressed-bytes": 59052,
          "target-type": "vee_statistic",
          "target-index": "vee"
        }
      ]
    }
  ],
   "schedule": [
    {
      "operation": {
        "operation-type": "delete-index"
      }
    },
    {
      "operation": {
        "operation-type": "create-index"
      }
    },
    {
      "operation": {
        "operation-type": "cluster-health",
        "request-params": {
          "wait_for_status": "green"
        }
      }
    },
    {
      "operation": {
        "name": "query-match-all",
        "operation-type": "search",
        "body": {
          "query": {
            "match_all": {}
          }
        }
      },
      "clients": 8,
      "warmup-iterations": 1000,
      "iterations": 1000,
      "target-throughput": 100
    }
  ]
}

when I try to run this track with just one index devices I succeed. Yet, when adding multiple indices, I fail. It seems like I am doing something wrong when either defining multiple indices, or multiple corpora.
Mapping files are in the same directory as track.json
Any help will be appreciated


(Dimitrios Liappis) #2

Hi,

I was almost about to type that I can't reproduce this but I forgot the search part.

You are not specifying the index for the search operation. As per the docs:

index (optional): An index pattern that defines which indices should be targeted by this query. Only needed if the index section contains more than one index. Otherwise, Rally will automatically derive the index to use. If you have defined multiple indices and want to query all of them, just specify "index": "_all".

If you use "index": "_all" it will work with the multiple indices.

Agreed the error message is not good. It should mention which operation is missing this.

I will open a ticket to address it.

Regards,
Dimitris


(Emil Mirzayev) #3

Thank you @dliappis !
I changed the config file, and my tests were able to run.
However, I noticed another "problem". The reason writing it in quotes, is I am not sure if it is intended or not.
When I run the tests, even the samples from esrally (percolator) itself, I never see CPU statistics.
This is what I got from my last custom test:

Lap Metric Task Value Unit
All Total indexing time 0.0969833 min
All Min indexing time per shard 0 min
All Median indexing time per shard 0 min
All Max indexing time per shard 0.05175 min
All Total merge time 0.0579 min
All Min merge time per shard 0 min
All Median merge time per shard 0 min
All Max merge time per shard 0.0495833 min
All Total refresh time 0.119917 min
All Min refresh time per shard 0 min
All Median refresh time per shard 0 min
All Max refresh time per shard 0.0729833 min
All Total flush time 3.33333e-05 min
All Min flush time per shard 0 min
All Median flush time per shard 0 min
All Max flush time per shard 1.66667e-05 min
All Total Young Gen GC 2.291 s
All Total Old Gen GC 0 s
All Store size 0.182953 GB
All Translog size 0.0181792 GB
All Heap used for segments 1.2836 MB
All Heap used for doc values 0.254734 MB
All Heap used for terms 0.88253 MB
All Heap used for norms 0.00561523 MB
All Heap used for points 0.0414066 MB
All Heap used for stored fields 0.0993118 MB
All Segment count 179
All error rate bulk 0 %
All Min Throughput query-match-all 99.96 ops/s
All Median Throughput query-match-all 99.99 ops/s
All Max Throughput query-match-all 100 ops/s
All 50th percentile latency query-match-all 26.4198 ms
All 90th percentile latency query-match-all 33.669 ms
All 99th percentile latency query-match-all 61.1824 ms
All 99.9th percentile latency query-match-all 79.9844 ms
All 100th percentile latency query-match-all 104.357 ms
All 50th percentile service time query-match-all 26.195 ms
All 90th percentile service time query-match-all 33.386 ms
All 99th percentile service time query-match-all 60.9567 ms
All 99.9th percentile service time query-match-all 78.1752 ms
All 100th percentile service time query-match-all 96.4109 ms
All error rate query-match-all 0 %

Am I doing something wrong here?
I have a simple config file, didin't tweak it too much.


(Dimitrios Liappis) #4

Hi,

Which pipeline are you using? If benchmark-only as mentioned in the docs:

As Rally has not provisioned the cluster, results are not easily reproducable and it also cannot gather a lot of metrics (like CPU usage).


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.