How to index routing id based document in rally

Hi @thennamca,

unfortunately, Rally does not support request parameters for the bulk API. However, you can generate the action and meta-data line yourself and tell Rally about it. If you generate this line yourself, you can add the _routing parameter there, e.g.:

{ "index" : { "_index": "test", "_type": "type1", "_id": "1", "_routing": 2 } }

So instead of:

 {“firstname”:“first_name1”,“lastname”:“last_name1”,“client”:100,“customerID”:2}
 {“firstname”:“first_name2”,“lastname”:“last_name2”,“client”:100,“customerID”:2}
 {“firstname”:“first_name3”,“lastname”:“last_name3”,“client”:200,“customerID”:2}

your file with the source documents would look like this:

{ “index” : { “_index” : “test”, “_type” : “type1”, “_id” : “1”, “_routing : 2 } }
{“firstname”:“first_name1”,“lastname”:“last_name1”,“client”:100,“customerID”:2}
{ “index” : { “_index” : “test”, “_type” : “type1”, “_id” : “2”, “_routing : 2 } }
{“firstname”:“first_name2”,“lastname”:“last_name2”,“client”:100,“customerID”:2}
{ “index” : { “_index” : “test”, “_type” : “type1”, “_id” : “3”, “_routing : 2 } }
{“firstname”:“first_name3”,“lastname”:“last_name3”,“client”:200,“customerID”:2}

As I've mentioned, you also need to tell Rally that your source data contains the action and meta-data line already. E.g.:

{
  "name": "index-append",
  "operation-type": "index",
  "action-and-meta-data": "sourcefile",
  "bulk-size": 5000
}

The property action-and-meta-data is set to sourcefile which tells Rally that your data already contain this line. Hence Rally will not generate it on the fly (also see the track reference in the docs).

Btw, Rally will not consider the action and meta-data line in the bulk size or in any statistics. So the bulk size of 5.000 is really 5.000 documents (which is 10.000 lines in the source file: 5.000 documents and 5.000 action and meta-data lines).

I hope that helps.

Daniel