How to benachamr single decumnt indexing

[marek_duciuc]

Is this correct way how to index benchmark single document in rally?

  {
      "operation": {
        "name": "small",
        "operation-type": "bulk",
        "bulk-size": "1",
        "ingest-percentage": "1",
        "batch-size":"1000"
      },

[json (Jason Bryan)]

Marek, thank you for your inquiry. It is good to see you again! ingest-percentage is the overall percentage of the document corpus. In other words, for ingest-percentage: 1 to load a single document, there would need to be 100 documents in the corpus since 1 is 1% of 100.

Try using esrally subcommand create-track against a cluster after having indexed the single document. Please see the tutorial for the steps.

[marek_duciuc]

Thank you for answer. What I want is to ingest 100k documents but each as separate index request and then using different batch sizes to see improvement in indexing thruput by using batch instead of indexing by single doc. And finally I want to test best bulk size.

[json (Jason Bryan)]

I see and I clearly misinterpreted your question. I think you can pull off what you are trying to do with bulk-size and batch-size. Instruct Rally to read N documents from the corpus at once with the batch-size parameter. The bulk-size parameter maps to the number of documents sent to the cluster in a single bulk request, so it should provide what you want. Note you are still using the bulk API, which may or may not be in line with your use case.

You might also try executing a raw-request with N iterations in the schedule if your application uses the Elasticsearch Index API as opposed to the Bulk API. I think it would require a custom runner if you wanted to send unique documents this way.