# How to reuse already indexed data in next race?

**URL:** <https://discuss.elastic.co/t/how-to-reuse-already-indexed-data-in-next-race/124043>\
**Category:** Elasticsearch\
**Tags:** rally\
**Created:** [March 15, 2018, 8:33am UTC](https://discuss.elastic.co/t/how-to-reuse-already-indexed-data-in-next-race/124043 "2018-03-15T08:33:11Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![asp](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@asp](https://discuss.elastic.co/u/asp)\
**Post date:** [March 15, 2018, 8:33am UTC](https://discuss.elastic.co/t/how-to-reuse-already-indexed-data-in-next-race/124043/1 "2018-03-15T08:33:11Z")

</div>

Hi,

I would like to build up a custom track. In the end it should be like a lite version of [https://github.com/elastic/rally-eventdata-track](https://github.com/elastic/rally-eventdata-track). But more simplified and with our dataset and our main kibana dashboard queries.

I need to input some historical data first. I think this will take a long time, but this inserting is not what I would like to test in main scope. So I want to index historical data only **once** and would like to **reuse this already indexed data** for the following races.

How can I do this?

I found the parameter preserve-install which keeps the installation and the data below the races directory. But How can I reuse these instances in following races?

Thanks, Andreas

---

<div class="post-metadata">

**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\
**Post date:** [March 15, 2018, 8:57am UTC](https://discuss.elastic.co/t/how-to-reuse-already-indexed-data-in-next-race/124043/2 "2018-03-15T08:57:02Z")

</div>

Hi @asp,

admittedly, what you want to achieve is a bit tricky currently. IMHO you have two options:

- Ingest the data and reuse the cluster that contains the data (by specifying `--preserve-install=true` on the command line). The cluster will be in `~/.rally/benchmarks/races/$TIMESTAMP/rally-node-0/install` on each of the target nodes. Afterwards, you need to start the cluster yourself (instead of having Rally do it for you) by using `--pipeline=benchmark-only` (see the [docs](http://esrally.readthedocs.io/en/stable/recipes.html#benchmarking-an-existing-cluster) for details)
- Use snapshots: This also requires that you do an initial ingestion with `--preserve-install=true`. You'd then create a snapshot with the [snapshot API](https://www.elastic.co/guide/en/elasticsearch/reference/6.2/modules-snapshots.html#_snapshot). You need to store that snapshot then in a place where you benchmark infrastructure can access it (e.g. an internal Webserver or a private S3 bucket). For your track, you'd then need a [custom runner](http://esrally.readthedocs.io/en/stable/adding_tracks.html#custom-runners) that can restore that snapshot in the first step.

IMHO the second option is the cleaner approach because it allows you restore the data without the need to keep the cluster. We plan to [add support to restore from a snapshot](https://github.com/elastic/rally/issues/341) out of the box so this will be simpler in the future.

Daniel

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 12, 2018, 8:57am UTC](https://discuss.elastic.co/t/how-to-reuse-already-indexed-data-in-next-race/124043/3 "2018-04-12T08:57:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.