# Synced flush and recovery **URL:** \ **Category:** Elasticsearch\ **Created:** [September 10, 2015, 9:17pm UTC](https://discuss.elastic.co/t/synced-flush-and-recovery/29072 "2015-09-10T21:17:12Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![achow](https://avatars.discourse-cdn.com/v4/letter/a/a3d4f5/32.png) [@achow](https://discuss.elastic.co/u/achow)\ **Post date:** [September 10, 2015, 9:17pm UTC](https://discuss.elastic.co/t/synced-flush-and-recovery/29072/1 "2015-09-10T21:17:12Z")
I'm having issues with getting recovery to use sync markers to recover from local replicas during planned full cluster restarts. Recovering from "sealed indices" works for about half of replicas, then for whatever reason, the other half gets copies over the network. I'm currently running v1.6.2. My restart flow is as follows: Stop all indexing Execute a synced flush on all indices Disable shard allocation Rolling restart of all data nodes Enable shard allocation after ensuring all data nodes have joined the cluster When the cluster is fully recovered, restart indexing My suspicion is that as unallocated shards get reallocated, the allocator is unaware that some nodes have local copies of those shards and allocates to some other node, preventing local recovery. Watching cluster state corroborates this -- some nodes are allocated way more shards than others. Say we have a cluster with data nodes A, B, and C and an index a with 3 shards and 1 replica (we'll name shards a1p for primary shard 1 of index a, a0r for replica shard 0 of index a. At steady state, allocation for these shards will be { A: [a0p, a1r], B: [a1p, a2r], C: [a2p, a0r] }. After a rolling restart of A, B, and C in that order but before reenabling shard allocation, the allocation ends up being { A:[a0p, a1p], B: [a2p], unallocated: [a0r, a1r, a2r] }. After enabling shard allocation, a0r gets allocated on B, but of course B doesn't have that shard. B ends up copying it over the network from A. a0r could have been allocated to C (and is definitely preferable), but for whatever reason doesn't. Does this make sense at all, and if so, is this a known issue? Synced flushes definitely speed up recovery, and it's be great if we reused all local shards when reallocating. ---
**Author:** ![s1monw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s1monw/32/3637_2.png) [@s1monw](https://discuss.elastic.co/u/s1monw)\ **Post date:** [September 11, 2015, 7:15am UTC](https://discuss.elastic.co/t/synced-flush-and-recovery/29072/2 "2015-09-11T07:15:52Z")
> [@achow](#): > > Execute a synced flush on all indices when you do this what does the return look like. is it saying that all syncs where successful? Is the cluster green by that time? ---
**Author:** ![achow](https://avatars.discourse-cdn.com/v4/letter/a/a3d4f5/32.png) [@achow](https://discuss.elastic.co/u/achow)\ **Post date:** [September 11, 2015, 5:55pm UTC](https://discuss.elastic.co/t/synced-flush-and-recovery/29072/3 "2015-09-11T17:55:58Z")
Yes, synced flush succeeds for all shards and cluster is green. ---
**Author:** ![promiseofcake](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/promiseofcake/32/18365_2.png) [@promiseofcake](https://discuss.elastic.co/u/promiseofcake)\ **Post date:** [June 20, 2017, 7:54pm UTC](https://discuss.elastic.co/t/synced-flush-and-recovery/29072/4 "2017-06-20T19:54:28Z")
Running into similar behavior on 2.4.2. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [June 21, 2017, 1:29am UTC](https://discuss.elastic.co/t/synced-flush-and-recovery/29072/5 "2017-06-21T01:29:30Z")
Please start your own thread with more information 🙂 ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [June 21, 2017, 1:29am UTC](https://discuss.elastic.co/t/synced-flush-and-recovery/29072/6 "2017-06-21T01:29:32Z")