# Shard Awareness Question **URL:** \ **Category:** Elasticsearch\ **Created:** [September 22, 2021, 8:54pm UTC](https://discuss.elastic.co/t/shard-awareness-question/284941 "2021-09-22T20:54:20Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Dauni](https://avatars.discourse-cdn.com/v4/letter/d/85f322/32.png) [@Dauni](https://discuss.elastic.co/u/Dauni)\ **Post date:** [September 22, 2021, 8:54pm UTC](https://discuss.elastic.co/t/shard-awareness-question/284941/1 "2021-09-22T20:54:20Z")
Hello, We are running an ES 5.16 cluster in 3 DC's (total of 15 hosts, each host is 8 cpu / 32 GB ram and we have heap set to 16GB). DC1 = zone1 (1 master node, 6 data nodes) DC2 = zone2 (1 master node, 6 data nodes) DC3 = zone1 (master only) Reason for the DC3 with only a master is for redundancy in event we lose an entire DC (without having to enable multiple masters in each DC). On all 15 nodes we have: node.attr.rack: r1 Currently we have the awareness set to zone: cluster.routing.allocation.awareness.attributes: zone When we do maintenance work like OS patching or even bouncing, we do node by node to prevent any downtime. We will turn off shard allocation on the node we are working on so the shards that were on that node do not get reallocated. Then do the necessary work. After doing the work, we then enable shard allocation once the node is up and joins the cluster. Since our current setup will never have the Primary and Replica in the same DC (zone), it takes 30+ minutes for each shard in 1 index to sync up with the Primary. The 1 index in question we have 15 primaries and 1 replica set. The size of each shard is approximately 8GB and growing daily. We increased the max\_bytes\_per\_sec to 200mb, and the concurrent\_streams to 4. While this did help on the overall issue, it didn't help any on the individual shard level (still takes 30+ minutes to recovery from primary due to cross DC copying.) Is there anything we could do by changing awareness settings that could help out? My thought is that we will always have the potential for cross DC copying during recovery since the primary could be in the other DC regardless of how we setup the zones. We are not using snapshots, but if this is a viable option to speed up recovery...we could look into that aspect. Any ideas or suggestions would be appreciated. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 22, 2021, 10:33pm UTC](https://discuss.elastic.co/t/shard-awareness-question/284941/2 "2021-09-22T22:33:54Z")
Welcome to our community! 😃 > [@Dauni](#): > > ES 5.16 5.X has been [EOL](https://www.elastic.co/support/eol) for a few years now, you really need to upgrade as a matter of serious urgency. > [@Dauni](#): > > We are running an ES 5.16 cluster in 3 DC's Running an Elasticsearch cluster across DCs is not recommended or supported. Please upgrade and use CCR and/or CCS. > [@Dauni](#): > > Is there anything we could do Upgrade, upgrade, upgrade. There has been a lot of work done around shard allocation between 5.X and latest - 7.15. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [September 23, 2021, 8:13am UTC](https://discuss.elastic.co/t/shard-awareness-question/284941/3 "2021-09-23T08:13:49Z")
Just an update here, in newer versions you can run cross-DC clusters, please see [Resilience in larger clusters | Elasticsearch Guide [7.15] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/high-availability-cluster-design-large-clusters.html) ---
**Author:** ![Dauni](https://avatars.discourse-cdn.com/v4/letter/d/85f322/32.png) [@Dauni](https://discuss.elastic.co/u/Dauni)\ **Post date:** [September 23, 2021, 12:11pm UTC](https://discuss.elastic.co/t/shard-awareness-question/284941/4 "2021-09-23T12:11:49Z")
Thank you Mark for the information and link! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 21, 2021, 12:12pm UTC](https://discuss.elastic.co/t/shard-awareness-question/284941/5 "2021-10-21T12:12:07Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.