How to continuously keep indexing and querying in ES

Elastic Stack Elasticsearch
1

Hi team,

I am trying to run Rally to continuously index and query, but I see that all the challenges are limited to the documents or the size. Is there a way or a repository with the track to run the test for a long run. Suppose I am having a server with 128GB RAM, I would like to index upto 1.5 TB or more and also have read traffic. Is this possible with Rally, if yes what would be my ideal track and challenge for this scenario.
I found a git repo with similar kind of challenges, but there is no challenge with continuous indexing https://github.com/elastic/rally-eventdata-track
Can anyone help here.

2

As the dventdata track generates data you can create challenges that run concurrent indexing and querying over long time periods. I have used it to create challenges that have run for days.

3

@Christian_Dahlqvist:

Is there a challenge that I can use here, which is already present ? Similar to something here https://github.com/elastic/rally-eventdata-track

4

I would recommend creating your own challenge that is adjusted to fit the size and expected traffic of your cluster. It would be a good example to add to the event data track but does as far as I remember not currently exist.