Merging Deployments

Elastic Stack Elasticsearch
1

Hello Folks ,

Say on Cloud i have a deployment ABC & XYZ. I now want to merge these 2 deployments say XYZ to ABC instead of managing both the deployments separately .

Could someone share pointers or high level steps , do's/dont's of how above can be achieved ?

Thanks!!

2

Hello Team,

Just following up - wanted to check if someone can provide guidance on merging deployment XYZ into ABC on Cloud. Appreciate any pointers on the steps or best practices when you get a chance.

Thanks!!

3

Hello,

I don't think there is any documentation about this as it is not possible to merge deployments, so what you should do is to decomission one of the deployments and migrate everything to the other cluster.

How are you ingesting data in deployment XYZ? What do you want to migrate? Dashboards? Users?

4

Hi @Tortoise

Well, there is no real merging of clusters. You cannot add the data nodes from one cluster into another cluster that's not supported.

From there, you would have to tell us a little bit more about the size scale and a high level of data....

The first thing that would come to mind is snapshot and restore as that can bring over a lot of cluster state and the. Not all of the cluster state and other items but most of it templates, alerts etc if that is part of the scheme.

Do you have a commercial license?

Tell us a bit more perhaps we can help.

5

Thanks for the pointers @leandrojmp @stephenb

To answer your questions:

  • Ingestion: via Fleet-managed Elastic Agents + direct API, roughly [20-30] agents (Fleet Managed / Standalone via Container)
  • License: Yes, I have a commercial license and also opened a support case with Elastic - since this isn't a standard/documented scenario, I wanted to get input from the community as well in case anyone's tackled something similar.

Goal

Not just moving data - I want to fully retire ABC into XYZ: config layer (ILM, index/component templates, ingest pipelines, transforms), Kibana saved objects (dashboards, etc.), roles/spaces (recreated manually) and the Fleet-managed agents - without disturbing anything XYZ already has.

Plan I am working from - sanity-checking before I execute

  1. Config layer first, before any data restore. Pull ILM policies, index templates, component templates and ingest pipelines off ABC via their GET APIs and PUT them into XYZ, checking for name collisions against what's already there. A data stream won't restore into a cluster that doesn't already have a matching, data-stream-enabled index template, so this has to land before step 2.

  2. Data via snapshot/restore - scoped, not global. Snapshot ABC's data streams/indices (not the whole cluster) and restore into XYZ with 'include_global_state: false' and no 'feature_states' requested. My understanding is that restore only touches templates/pipelines/ILM/persistent settings when global state is included - with it off, the restore should be scoped to just the indices/data streams I name and shouldn't clobber anything on XYZ. Wanted to confirm that's actually how it behaves in practice and flag the one collision risk I can see: if ABC and XYZ happen to share an index or data stream name already, that's a separate problem restore won't quietly resolve for you.

  3. Transforms - recreate via '_transform' GET/PUT (stopped, pointed at the now-migrated source/dest indices on XYZ) rather than trying to carry over checkpoint state from ABC.

  4. Kibana saved objects (dashboards, visualizations, alerting rules) - Saved Objects export/import (NDJSON) with conflict resolution, not a '.kibana' snapshot restore. Since that's a system index/feature state, pulling it in via global state would step on XYZ's own Kibana config rather than merge cleanly.

  5. API keys - my understanding is these simply don't migrate between clusters; they're bound to the issuing cluster's security index. Planning to recreate them fresh on XYZ once the equivalent roles exist there.

  6. Roles/Spaces - recreating directly on XYZ, users remapped to the new role names. Not attempting to bring these over mechanically.

  7. Fleet/Agents - I see Fleet now has an agent-migration API (bulk and single-agent) that repoints an already-enrolled agent at a different cluster's Fleet Server without a manual re-enroll. Planning to use that instead of standing up new policies and re-enrolling everything by hand, assuming our version supports it.

What I still want a sanity check on

Is there any scenario where a data-only restore (global state off, no feature states) can still touch or clash with XYZ's existing system indices (security, Kibana, etc.) - or is that risk genuinely limited to when you include global state/feature states? And has anyone actually run a cutover like this end to end - particularly interested in gotchas around losing transform checkpoint state and how to sequence the Fleet agent migration against a hard cutover window without a gap in ingestion.

Thanks!!