I am just curious if there are any performance results available for any ESRally tracks. I searched on the internet but have not come up with anything. Here is the reason why I am looking for such data. I'd like to benchmark my setup with certain ESRally tracks, like dense_vectors, http_logs, and so on. I just want to have a feel of what others in the wild have tested and what kind of results they saw in terms of documents/s, ops/s, and latency for operations like index-append, knn-search, etc.. Not sure if we have a centralized repository somewhere that contains these types performance results (similar to MLPerf data by ML Commons. Benchmark MLPerf Storage | MLCommons V1.1 Results).
I am not looking for any specific performance related to CPU, memory, or storage. I just want to know if others have tested with whatever they have and I can extrapolate from there.
Thank you, @Kathleen_DeRusso ! You are awesome! Let me dissect the link you sent me. I will ask more questions if I don't understand something. Hope it's alright.
Hi @Kathleen_DeRusso , I have a couple of follow up questions if you don't mind.
What does "add-4g-1node-index-append" represent? Is that a single ES node and what resources does it have in terms of CPUs, memory, and disks?
For the graphs that show the nightly throughput, is that the maximum throughput or the median throughput? Also this throughput is from that single node?
I am trying to understand the testbed so I can get more context from the performance results. Hope you don't mind.
Hmm, you're right that we have all of this information in a summary report but the dashboards are a bit vague. I would assume that it's total but I'm going to tag this post with rally to get a definitive answer from someone more familiar with this/from the team that supports Rally!
Firstly the questions regarding the specs - on the index page we list the specs of the bare metal machines we use - Elasticsearch Benchmarks
Note that the hardware we are currently using is from 2017, so is fairly old, for our uses it is fine, though the benchmarks we are running are primarily meant for detecting regressions, not for showcasing the fastest Elasticsearch can go.
Some of the benchmarks (not dense_vector) run on cloud instance, in which case we should have the details of the hardware used on the benchmark page, otherwise if not stated it should be the bare metal machines listed on the main page.
add-4g-1node-index-append
the 4g-1node part refers to, as you guessed, a single ES node, with 4G heap. index-append is the action within the track (ie this one here.)
For the graphs that show the nightly throughput, is that the maximum throughput or the median throughput? Also this throughput is from that single node?
The charts on the page show mean throughput (or latency, depending on the operation) for the whole of the run (or, if for searching, after a certain number of iterations are completed, so that we only take metrics from Elasticsearch when warmed up, see here for an example of where we set warmup vs normal iterations). The results on the benchmark page isn't specifically supposed to be showing "this is the fastest Elasticsearch can go", but rather the results from a static hardware setup, with the primary goal to be to help us detect regressions or improvements in Elasticsearch. As such, we are showing relative performance. If you were to go and run on more powerful hardware, then you would likely see better performance.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
