Stuggling for a TLDR: can someone give a rough sense how much CCS has changed between 5.5 and 6.5, specifically around how the number of indices & shards in the remotes can impact the CCS nodes? Or, any caveats to mind when upgrading CCS from 5.5 to 6.5?
While reading this elastic blog post, the linked pull request challenged my understanding of how CCS works. I thought the interplay between CCS nodes and its remotes was very similar to a normal elasticsearch client with the primary difference being that CCS nodes could go through a reduction phase. From this PR a couple of weeks ago, I thought this was how CCS was designed & implemented initially:
This will be used in cross-cluster search when reduction will be
performed locally on each cluster. The CCS coordinating node will send
one search request per remote cluster involved and will get one search
response back from each one of them. Such responses contain all the info
to be able to perform an additional reduction and return results back
to the user.
I also stumbled on this removal in the 6.5 docs which has me questioning my understanding as well as how much is changing under the hood.
Thanks and apologies for the vague ask!