Hi Team,
We are planning to use ElasticSearch and to test our expected load we want to run some load test. I was looking at custom track options and see that bulk operation is used for indexing documents. However, we won't be able to use bulk api's in production, and would be using put api's to create/update document. Can i simulate this using Rally? Is this equivalent to using bulk with bulk-size=1? If not, are there any out of the box operation to achieve indexing of single document?
Technically a bulk operation with a bulk size of 1 is not the same as indexing a single document. The bulk payload needs additional information in a ndjson format, and that in itself will be different than if you sent only one document. Rally does not have an operation that supports indexing a single document out of the box.
Authoring a custom runner can get your what you want. This is a performance critical codepath, and it is easy to generate results that are not accurate if something is messed up in a custom runner.
That said, bulk requests are much more efficient than a single PUT to create a document. There is a lot of network layer overhead to sending single requests that can be alleviated with a bulk operation. It is likely you will see bottlenecks in PUT requests in your application, before you get close to exhausting Elasticsearch's ability to index bulk requests.
Got it. Thanks you! We will eventually move to bulk however, for our initial launch we are thinking of indexing single document. I will look into other options for load testing. We are getting live events which will be inserted/indexed into ElasticSearch, and customer would query elasticsearch for this information. We cannot wait for batching the events hence we might index single document.
Just a small FYI, for one off API operations, apart from using a custom runner, the raw request operation can also be useful.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.