Elasticsearch rolling restart without indexing down time

Elastic Stack Elasticsearch
1

We run an elasticsearch cluster 7.17, with 3 data nodes and 3 master nodes. The use case is for monitoring with elasticAPM.

We follow official documentation at Full cluster restart upgrade | Elasticsearch Guide [7.17] | Elastic to update our elaticsearch cluster.

It works fine, but at the end, we have found data miss on apm indices!

In the documentation, you say to disable allocation before removing the node, hence there is no primary shard, hence APM cannot write data.

So, is there a good practice to update a cluster without data loss? Without changing architecture of course (no queue between APM and elasticsearch).

2

Are you sure you are not referring to the rolling upgrade instructions? The full cluster restart would result in the cluster being unavailable for some period of time.

Do you have replica shards configured or are you running with only primary shards?

3

well about replica, I try to understand, the 1st instruction is to ... disable replication, right ?

PUT _cluster/settings
{
  "persistent": {
    "cluster.routing.allocation.enable": "primaries"
  }
}