Is it possible to create a loop around a challenge

@matt.veitas is correct with the loop idea. In my answer in the thread "Increase data size in Rally existing tracks" I have provided an example how loops can be used.

If you want to run multiple iterations of the same benchmarking trial (without restarting Elasticsearch) you can also use the command line parameter --laps, e.g. --laps=12 will run the same benchmark twelve times. Rally will not restart Elasticsearch in between and will (usually) recreate the index at the beginning of each iteration (lap).