How to limit the dataset size of elastic/security ESRally track

Elastic Stack Elasticsearch
1

I am running esrally elastic/security track on ESRally version 2.7.0.
By default, it downloads the following datasets, with total size of 128GB. but I only need ~30-50GB input dataset size.

rally@benchmark-bqfd7:~/.rally/benchmarks/data$ du -h
21G     ./elastic/security/filebeat-security
32G     ./elastic/security/packetbeat-security
33G     ./elastic/security/auditbeat-security
11G     ./elastic/security/winlogbeat-security
130M    ./elastic/security/generated/99cd64d93e95084bb8bc7eea08cc1dbf
130M    ./elastic/security/generated
32G     ./elastic/security/metricbeat-security
128G    ./elastic/security
128G    ./elastic
128G    .

command used:

esrally race --track=elastic/security --track-params="wait_for_status:yellow,number_of_shards:2,number_of_replicas:0,max_generated_corpus_size:100MB" --pipeline=benchmark-only --target-hosts=https://es-master:9200 --challenge=security-indexing-querying --client-options=basic_auth_user:rally,basic_auth_password:changeme,timeout:240,use_ssl:true,verify_certs:false,ca_certs:/rally/cacert.pem --report-format=csv --kill-running-processes

I also tried the following track_params, but still downloaded data size remains the same.
max_total_download_gb:1
max_generated_corpus_size:1GB
force_data_generation:true

Thanks in advance!

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.