# Benchmarking sequence of operations **URL:** \ **Category:** Elasticsearch\ **Tags:** rally\ **Created:** [April 25, 2018, 7:50pm UTC](https://discuss.elastic.co/t/benchmarking-sequence-of-operations/129557 "2018-04-25T19:50:32Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![tanenblatt](https://avatars.discourse-cdn.com/v4/letter/t/eada6e/32.png) [@tanenblatt](https://discuss.elastic.co/u/tanenblatt)\ **Post date:** [April 25, 2018, 7:50pm UTC](https://discuss.elastic.co/t/benchmarking-sequence-of-operations/129557/1 "2018-04-25T19:50:32Z")
Being relatively new to using Rally, maybe I'm missing something, but: Our system is using Elasticsearch to select some documents, then uses data from those documents to select some other documents, and then uses data from those documents to select another set of documents, etc., until we reach some end state. This may require three or four levels of queries. We are trying to understand how different implementations of these queries might impact performance, so Rally seems to be the perfect tool, but we're not seeing how we can run one operation after another within a challenge, instead of one operation _n_ times, then the next operation _n_ times. The concern is how these two different approaches would be impacted by caching: it seems running a sequence of 4 operations _n_ times would give us different results than each of the 4 operations _n_ times individually. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 26, 2018, 6:22am UTC](https://discuss.elastic.co/t/benchmarking-sequence-of-operations/129557/2 "2018-04-26T06:22:30Z")
For a complex series of operations like that with dependencies, I would probably recommend treating the full sequence of calls as a single operation. You can do that by creating a [custom runner](http://esrally.readthedocs.io/en/stable/adding_tracks.html?highlight=custom%20runner#custom-runners) that executes the full series of operations and reports on this as a single unit. I did this when I wanted to [simulate Kibana queries in my eventuate track](https://github.com/elastic/rally-eventdata-track/blob/master/eventdata/runners/kibana_runner.py). If you need to have control over how the tasks for this runner are created, you can handle this through [a custom parameter source](http://esrally.readthedocs.io/en/stable/adding_tracks.html?highlight=custom%20runner#custom-parameter-sources). Here is a link to what I used [for my Kibana runner](https://github.com/elastic/rally-eventdata-track/blob/master/eventdata/parameter_sources/elasticlogs_kibana_source.py). ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [April 27, 2018, 6:35am UTC](https://discuss.elastic.co/t/benchmarking-sequence-of-operations/129557/3 "2018-04-27T06:35:07Z")
I think Christian has described quite well how you can implement this currently. I have raised [https://github.com/elastic/rally/issues/486](https://github.com/elastic/rally/issues/486) in Rally so maybe we can support this out of the box at some point. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 27, 2018, 7:04am UTC](https://discuss.elastic.co/t/benchmarking-sequence-of-operations/129557/4 "2018-04-27T07:04:16Z")
As the second phase of queries depend on the results from the first, I suspect a custom runner is the only way to go for this scenario. Being able to sequence operations is however a very useful addition for cases when they can all be driven off the same configuration. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [April 27, 2018, 8:27am UTC](https://discuss.elastic.co/t/benchmarking-sequence-of-operations/129557/5 "2018-04-27T08:27:41Z")
> [@Christian\_Dahlqvist](#): > > As the second phase of queries depend on the results from the first, I suspect a custom runner is the only way to go for this scenario. Yes, if you need to feed data from one query to the next, I definitely agree. ---
**Author:** ![tanenblatt](https://avatars.discourse-cdn.com/v4/letter/t/eada6e/32.png) [@tanenblatt](https://discuss.elastic.co/u/tanenblatt)\ **Post date:** [May 1, 2018, 1:42pm UTC](https://discuss.elastic.co/t/benchmarking-sequence-of-operations/129557/6 "2018-05-01T13:42:21Z")
Actually, there are cases where being able to sequence operations would be extremely helpful for us. One way we'd use that would be to run specific sets of canned sequences against a standard data set, to compare against previous sequences that provide the same results (In other words, we might have some set of operations Oa: { op1, op2, op3 } that are replaced by the semantically equivalent Ob: { op1, op2a, op2b, op3 }). We would do this in our testing, to assure that we haven't introduced any delays. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 29, 2018, 1:42pm UTC](https://discuss.elastic.co/t/benchmarking-sequence-of-operations/129557/7 "2018-05-29T13:42:26Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.