# Elastic benchmark changes

**URL:** <https://discuss.elastic.co/t/elastic-benchmark-changes/48186>\
**Category:** Elasticsearch\
**Created:** [April 22, 2016, 3:45pm UTC](https://discuss.elastic.co/t/elastic-benchmark-changes/48186 "2016-04-22T15:45:57Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ceilingfish](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ceilingfish/32/8790_2.png) [@ceilingfish](https://discuss.elastic.co/u/ceilingfish)\
**Post date:** [April 22, 2016, 3:45pm UTC](https://discuss.elastic.co/t/elastic-benchmark-changes/48186/1 "2016-04-22T15:45:57Z")

</div>

Hi,  
I was looking at the benchmarks for elasticsearch you chaps publish at [https://benchmarks.elastic.co/index.html](https://benchmarks.elastic.co/index.html)

It seems that around the end of February 2016 there's a big change in the performance characteristics of the tests. Is there a specific change that was made around then? If so what was it?

Regards  
Tom

---

<div class="post-metadata">

**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\
**Post date:** [April 23, 2016, 12:23pm UTC](https://discuss.elastic.co/t/elastic-benchmark-changes/48186/2 "2016-04-23T12:23:48Z")

</div>

Hi Tom,

if you hover the graphs at this point in time, it says that we changed the benchmark data set from a log based benchmark to a geonames based benchmark. So this is totally unrelated to changes in Elasticsearch.

Note that the scripts that produce the benchmark results are not available as open source (it is also a bit hard to set them up). That's why we've built [Rally](https://github.com/elastic/rally). It is available as open source, actively maintained by us and much easier to set up. Also the benchmark data sets are openly available so you can reproduce the benchmark results yourself.

We publish similar benchmark results for the geonames data set produced by Rally at [https://elasticsearch-benchmarks.elastic.co/](https://elasticsearch-benchmarks.elastic.co/) - Some of the graphs are a bit bumpy before mid of April. This is because Rally is rather new but we still actively look for problems in the benchmark driver and improve it continuously. In summary, I'd say starting with mid of April the data are quite reliable.

We're also about to add new data sets (logging, full text etc.) for Rally but we will not mix different benchmark data sets but rather create new dashboards for them. For more details see also the [release blog post for Rally](https://www.elastic.co/blog/announcing-rally-benchmarking-for-elasticsearch) and don't hesitate to ask questions about Rally here in the Elasticsearch forum.

Daniel

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:56pm UTC](https://discuss.elastic.co/t/elastic-benchmark-changes/48186/3 "2017-07-05T22:56:49Z")

</div>