Benchmark for existing cluster

[Thennarasu_Subramani (Thennarasu Subramaniyan)]

Hi All

Greate day

I want to take a benchmark for my existing cluster,i am totally confusing where i have to configure my elastic search details,i am having ES server in EC2 instance ,here my configuration is one master node ,one data node and one replica ,here i just indexed 1 million of data ,i want to see this benchmark ,i already configured esrally,here defalut track(geoname) based banchmark working perfect ,but i want to take my indexed data benchmark ,how i have to configure ,what are the changes i have to do in configuration level.

can you please any one share example of configuration,or please guide me how to do it

i just refered esrally custom track here i am not get full information ,any one please share me step by step

Thanks
Thennarasu.S

[danielmitterdorfer]

Hi @Thennarasu_Subramani,

For the following explanation I assume that you have created a custom track in ~/.rally/benchmarks/tracks/all-my-custom-tracks and your track is called my-track.

To just run this track - say against Elasticsearch 5.5.0 - on your local machine you'd need to run:

esrally --track-repository=all-my-custom-tracks --track=my-track --distribution-version=5.5.0

If you want to run it against a remote machine or a cluster of multiple machines, use the --target-hosts parameter. E.g.:

esrally --track-repository=all-my-custom-tracks --track=my-track --pipeline=benchmark-only --target-hosts:10.17.23.11:9200,10.17.23.12:9200

You can find a complete walk-through in the docs.

Daniel

[Thennarasu_Subramani (Thennarasu Subramaniyan)]

Hi @danielmitterdorfer
yes i am tring to create custom track,but i dont know how to refer my ec2 elastic search server and my index,can you provide those steps .
I am using elasic search 2.4.5 version please guide me step by step how to conifugure my index to rally track

[danielmitterdorfer]

Hi @Thennarasu_Subramani,

unfortunately, the more we talk, the more I have the impression that we still talk past each other.

I was under the impression that you have already written your track but this does not seem to be the case.

So your first step is to write a track (I have repeatedly pointed you to the corresponding docs). Apart from creating a track.json, you need to prepare a file containing all the data that you want to bulk index. How you create this file is out of the scope of Rally. You can write it manually, you can write a generator, you can extract data with a script from an existing cluster. How you do it is totally up to you. But in the end, you should have a file that is compatible with Elasticsearch's bulk index format.

Let's have a look at how this file looks for the geopoint track (these are just the first few lines):

{"location" : [-0.1485188, 51.5250666]}
{"location" : [-0.1479949, 51.5252071]}
{"location" : [-0.1458559, 51.5289059]}
{"location" : [-0.1498551, 51.5282564]}
{"location" : [-0.1487043, 51.5254843]}

Put this stuff in ~/.rally/benchmarks/data/YOUR_TRACK_NAME_HERE/documents.json and create a track.json file in the right directory and reference your documents (please see how to do this in the docs).

Daniel