We are planning to use ElasticSearch and to test our expected load we want to run some load test. I was looking at custom track options and see that
bulk operation is used for indexing documents. However, we won't be able to use bulk api's in production, and would be using
put api's to create/update document. Can i simulate this using Rally? Is this equivalent to using bulk with
bulk-size=1? If not, are there any out of the box operation to achieve indexing of single document?
Technically a bulk operation with a bulk size of 1 is not the same as indexing a single document. The bulk payload needs additional information in a ndjson format, and that in itself will be different than if you sent only one document. Rally does not have an operation that supports indexing a single document out of the box.
Authoring a custom runner can get your what you want. This is a performance critical codepath, and it is easy to generate results that are not accurate if something is messed up in a custom runner.
That said, bulk requests are much more efficient than a single
PUT to create a document. There is a lot of network layer overhead to sending single requests that can be alleviated with a bulk operation. It is likely you will see bottlenecks in
PUT requests in your application, before you get close to exhausting Elasticsearch's ability to index
Got it. Thanks you! We will eventually move to bulk however, for our initial launch we are thinking of indexing single document. I will look into other options for load testing. We are getting live events which will be inserted/indexed into ElasticSearch, and customer would query elasticsearch for this information. We cannot wait for batching the events hence we might index single document.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.