Rally outputs Index Size, and Totally Writen. For the Geonames benchmark it gives
Index size [GB] | 3.31185
Totally written [GB] | 21.1952
What exactly is Totally written?
Rally outputs Index Size, and Totally Writen. For the Geonames benchmark it gives
Index size [GB] | 3.31185
Totally written [GB] | 21.1952
What exactly is Totally written?
Hi @eugene_miretsky,
"Totally written" is the number of bytes that have been written to disk when indexing data. As Lucene starts out with smaller segments which are then merged to larger ones this number is greater than the index size.
Depending on your platform, Rally tries to determine this number just for the Elasticsearch process (this should work on Linux) or it falls back to using the difference between the totally system-wide written bytes before and after the benchmark (this is usually the case on Mac OS X). So the latter approach is less accurate but should still give you a ballpark estimate.
We use psutil to determine these numbers. If you're interested in more details, here are the docs:
Daniel
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.