When I upgrade from 5.x to 7.x, I find that I cannot upgrade directly and need to reindex all index data. Why not update the index data automatically during the upgrade?
There's been quite a lot of changes between those versions. Different field types, data structures and more. So it's not "simple" to just do it automatically and expect everything to work.
But if these things are not resolved, the best solution for a cluster upgrade is:
Create a new cluster
Enable dual-write and manual historical data synchronization
This is not user friendly
As opposed to what?
if there was an automatic upgrade system, what should happen to data that cannot be reindexed due to changes? Or data that errors out? How do we handle conflicts and changes?
Take a look at Upgrade Assistant | Kibana Guide [7.14] | Elastic instead.
Index data will not be updated. What contents will be updated when es major version is upgraded?
You mean what indexed data? Only segments that undergo merges after the upgrade, as they will take the new Lucene version.
There's probably also some system indices that will have auto upgrades applied too.
Elasticsearch uses Lucene and each major version generally corresponds to a new major version of Lucene. One limitation of Lucene with respect yo backwards compatibility is that only the previous major version can be read. Elasticsearch 7.x can therefore not read indices created in Elasticsearch 5.x which makes it impossible to upgrade directly. On top of this you have other breaking changes for each major Elasticsearch release that need to be handled or migrated and adds complexity.
If the index template is not compatible with the new version,the template content will be cleared?
Now that I think about the ES cluster upgrade, I can see why everyone has to manually import data into the new cluster when upgrading to a larger version.
If you follow along and upgrade to each new major version and do not need to make large version jumps it is less of a problem and quite well supported.
This is difficult, because the production environment will not be updated with the release of ES.
It is very rare for clusters to upgrade with every release, but it generally makes sense to follow along periodically so that you stay at most one major release behind. Elasticsearch 6.0 was released in November 2017 which is quite a long time ago. Elasticsearch 7.0 has been available for over 2 years as it was released in April 2019.
You mean with the ES upgrade, our application adjusts its service code with whom es upgrade?