# Possible host awareness when doing shard relocation? **URL:** \ **Category:** Elasticsearch\ **Created:** [January 25, 2018, 6:14am UTC](https://discuss.elastic.co/t/possible-host-awareness-when-doing-shard-relocation/116973 "2018-01-25T06:14:22Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![zjxian](https://avatars.discourse-cdn.com/v4/letter/z/77aa72/32.png) [@zjxian](https://discuss.elastic.co/u/zjxian)\ **Post date:** [January 25, 2018, 6:14am UTC](https://discuss.elastic.co/t/possible-host-awareness-when-doing-shard-relocation/116973/1 "2018-01-25T06:14:22Z")
We're running a cluster that each host (physical machine) has 2 es nodes running. On each host, 1 node (hot ) uses the SSD as storage while another ( stale ) uses SATA as storage. We are using shard allocation filter to control the index, so that recent indices are kept on hot nodes and older indices are kept on stale nodes. When some hot indices become "old", we change shard allocation filters so that es cluster automatically relocate them from hot to stale. However, the relocation is quite random. Suppose 3 hosts, H1 to H3. What I hope to see, is that shards on hot\_node\_H1 is relocate to stale\_node\_H1, and so on. But usually shards on hot\_node\_H1 is relocate to stale\_node\_H3, and cause what I think unnecessary network flows. So is there any related config to achieve this, so that in my scenario the relocation may be more efficient ? ---
**Author:** ![s1monw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s1monw/32/3637_2.png) [@s1monw](https://discuss.elastic.co/u/s1monw)\ **Post date:** [January 26, 2018, 10:05am UTC](https://discuss.elastic.co/t/possible-host-awareness-when-doing-shard-relocation/116973/2 "2018-01-26T10:05:52Z")
ok lemme ask some questions. - `stale_node_H3` and `hot_node_H3` is the same physical machine? - given my previous questions answer is `yes` why would you wanna do this can't you just leave it there? - if you move from any given node X to any given node Y we push stuff over the network you mean it could be more efficient to push it through loopback on the same node, correct? - if you already prevent the allocation on `hot_node_H3` via allocation filters why don't you use these filters to advice the system to move to `stale_node_H3`? - also if you have 2 nodes running on the same physical HW you should also set `cluster.routing.allocation.same_shard.host: true` to tell elasticsearch to not place any replicas on the same physical machine. If you do that the relocation won't work to the same physical HW. simon ---
**Author:** ![zjxian](https://avatars.discourse-cdn.com/v4/letter/z/77aa72/32.png) [@zjxian](https://discuss.elastic.co/u/zjxian)\ **Post date:** [January 26, 2018, 4:17pm UTC](https://discuss.elastic.co/t/possible-host-awareness-when-doing-shard-relocation/116973/3 "2018-01-26T16:17:51Z")
> [@s1monw](#): > > ok lemme ask some questions. > > - `stale_node_H3` and `hot_node_H3` is the same physical machine? > > Yes. Each host in my cluster has both SSD and SATA disks. Hot node uses SSD as storage, which is relatively small (240GB each node), stale node uses 3x6TB SATA disk (each node). > Currently the cluster uses 3 hosts ( 6 data nodes, 3 hot, 3 stale ), and a dedicated host as master node. > The indices are generated on a monthly basis. The storage spaces on all hot nodes may hold 2~3 month, but we're planning for long term (18~24 month). So we have to move indices from SSD to SATA disks. > > - given my previous questions answer is `yes` why would you wanna do this can't you just leave it there? > > As stated above, the SSD storage is not enough for long term. > > - if you move from any given node X to any given node Y we push stuff over the network you mean it could be more efficient to push it through loopback on the same node, correct? > > Yes. The network between each physical machine in our cluster is relatively poor, just 1Gbps, and is quite problematic to improve. So we're thinking transferring data through loopback may speed things up. > > - if you already prevent the allocation on `hot_node_H3` via allocation filters why don't you use these filters to advice the system to move to `stale_node_H3`? > > Maybe there is some mis-understanding. Currently we're using allocation filters to advice the system to move all shards that previously on all the hot node, to all the stale nodes. > The problem is, the cluster consists of several hosts. And after using allocation filter (exclude all hot nodes and include all stale nodes), for example the shards previous on hot\_node\_H3 may be relocated to stale\_node\_H2, which is the stale node on another host machine. And data is transferred through the 1Gps network. > > ![1](https://us1.discourse-cdn.com/elastic/original/3X/d/0/d0918fc05d478f3ff34cb38d4f07d1b7b6af84cb.PNG) > > ES seems quite random when choosing the destination.... > > - also if you have 2 nodes running on the same physical HW you should also set `cluster.routing.allocation.same_shard.host: true` to tell elasticsearch to not place any replicas on the same physical machine. If you do that the relocation won't work to the same physical HW. > > Replica setting is currently not our concern. The problem is we can't find a way to coordinate “shard relocation" in index relocation process. > > simon Thank you for your reply. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 23, 2018, 4:18pm UTC](https://discuss.elastic.co/t/possible-host-awareness-when-doing-shard-relocation/116973/4 "2018-02-23T16:18:10Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.