How to reindex data in Elasticsearch without actually changing index names?

Yes, you would need to do it for all indices.