You're generally going to want to have a read over the breaking changes documentation and run the elasticsearch-migration plugin on your 2.4 cluster before proceeding. Many of the questions you've asked are in that documentation and will be flagged by the migration checker as well. For example:
- There were some breaking changes in the mappings
- The search syntax changed some
A couple other points:
- Indices created in 2.4 can be read/written in 5.x (but not indices created prior to 2.0, even if they're in your 2.x cluster)
- You can run multiple versions on the same machine, but not (at least not easily) if you're installing from deb/rpm. You can run from a few zip/tar packages, just watch out that you set up different ports for them and don't try to form a cluster between the two. Also, FYI we also started releasing docker images if that's more your thing
- There's a deprecation log that you can turn on to look for queries that you may be using which are deprecated and may be removed in 5.x. The upgrade assistant can also help you turn this on.
You may want to watch our webinar on upgrading to 5 and has a demo of the migration assistant
As to the question "why upgrade to 5.x," I answered a similar question back when we released 5.0. Since I wrote that, we released 5.1 through 5.6, which also include a huge number of improvements, from cross-cluster search to field collapsing to various optimizations in the query execution like this to new "range" field types you can use to store an entire range of values (IP ranges, date ranges, numeric ranges) and a variety more.