Why should I upgrade to 5.0?

This is probably going to seem daunting due to the size, but the 5.0.0 release notes are at https://www.elastic.co/guide/en/elasticsearch/reference/current/release-notes-5.0.0.html and release notes for all of the 2.x versions are at https://www.elastic.co/guide/en/elasticsearch/reference/2.4/es-release-notes.html . The size of these release notes should be an indication of how much further things have come in Elasticsearch since the 1.x versions. We do wrap up some of the benefits in our release blog posts, which you can find at the following locations

Regarding the "are the benefits worth the pain / we don't use (m)any of these features," only you can decide the value that these may provide. I work for Elastic, and I'm inherently biased towards answering "of course it's worth upgrading! Look at all the great things that are there!" :slight_smile:

But in all seriousness, there are likely at least a few compelling reasons to upgrade. I'll leave it to you to decide if any of these are "worth it" to you:

  • You'll obviously never use a feature that you don't have access to, so saying "we don't use (m)any of these features" may be at least partially because they're not available to your developers/users. You may be using other datastores that you could simplify architectures on by using a later version of Elasticsearch instead, for example. We've seen many users get a lot of additional value through these features and deploy Elasticsearch in entirely new use cases as a result.
  • Resiliency is a hard thing to quantify, but if you're running 1.6 we can pretty authoritatively say that you stand a significantly increased risk of losing data at some point. See all of the items listed as 2.0-5.0 on our resiliency page for examples. Whether losing data in Elasticsearch is a big deal to you or not obviously depends on your use case(s) and what it means to recover that data. Some users have sophisticated methods to avoid some of these scenarios, but some are quite difficult to avoid and you may be setting yourself up for a 3am pager of a production outage, for example. What that means to your business obviously is up to you. 5.0 is so much better at resiliency than 1.6 that maybe you/your management would decide to use it for other reasons if you had the latest/greatest. If you're running a "large installation," I'd think that this reason alone may be very compelling.
  • With all of the performance benefits -- especially if you're storing numbers (including dates, metrics, IP addresses, or geo locations) though generally across the board -- you could benefit in reduced storage costs, CPU usage, and memory usage.
  • If your data is time-based, you could use the _rollover and/or _shrink APIs in 5.0 to optimize your shard utilization, which can yield improved performance and lower heap usage
  • If you're at all concerned with security, 2.0 is significantly better and 5.0 is significantly better than that. This means lower chances of somebody using Elasticsearch as an attack vector for the other machines in your corporation, for example.
  • Shortly, 1.6.0 will be end-of-life. It's generally not good to be running end-of-life software.
  • I'll add a bit of a plug for our company here as well: with 5.0, we have a nice set of features in our x-pack. It plugs nicely in to provide a variety of additional features, including security, alerting, monitoring, and graph functionality.

If you're running a large installation in an company, you'll ultimately need to consider how important Elasticsearch is to your business and what would happen if it went away all of a sudden due to resiliency issues or somebody used it as an attack vector to other systems you owned. That may be some indication as to whether you'd want to upgrade or not. The new features that may mean you could simplify or get additional benefit out of business/technical processes may be another reason. The performance benefits may be yet another.

How much these mean to you is ultimately going to be up to you, but I do hope you decide it's worth it!