Rally report - what does "Totally Written" mean?

Rally outputs Index Size, and Totally Writen. For the Geonames benchmark it gives
Index size [GB] | 3.31185
Totally written [GB] | 21.1952

What exactly is Totally written?

Hi @eugene_miretsky,

"Totally written" is the number of bytes that have been written to disk when indexing data. As Lucene starts out with smaller segments which are then merged to larger ones this number is greater than the index size.

Depending on your platform, Rally tries to determine this number just for the Elasticsearch process (this should work on Linux) or it falls back to using the difference between the totally system-wide written bytes before and after the benchmark (this is usually the case on Mac OS X). So the latter approach is less accurate but should still give you a ballpark estimate.

We use psutil to determine these numbers. If you're interested in more details, here are the docs:

Daniel