Is there any way to check the relocation progress of shards?

wajika · March 15, 2022, 10:44am

I plan to migrate the data in a cluster with three nodes to another cluster. After I add the exclusion parameter (cluster.routing.allocation.exclude._ip) to the cluster, check the status of shards through /_cat/shards It was RELOCATING, but for a day, I kept refreshing and repeatedly executed /_cat/shards, and all the shards in the RELOCATING state did not change.

How to know what's going on behind the RELOCATING state? How much has it been copied?

Christian_Dahlqvist · March 15, 2022, 10:46am

Have you looked at the cat recovery API?

wajika · March 15, 2022, 10:55am

Thanks, this command really helped me, do I just need to focus on these three fields next?
'files / files_recovered / files_percent'

Is there any way to know why copying is so slow or how to make it faster?

gist.github.com

https://gist.github.com/wajika/ee0b1cec1a1906d38f901b54c1ea56e6

shards_relocation

prevented by default
index                                                 shard time  type           stage source_host    source_node target_host    target_node repository snapshot files files_recovered files_percent files_total bytes       bytes_recovered bytes_percent bytes_total  translog_ops translog_ops_recovered translog_ops_percent
abp_auditlogs-000001                                  2     8.6h  peer           index 192.168.10.26  es-node01   172.20.134.162 es-node002  n/a        n/a      97    95              97.9%         97          1067697160  757331233       70.9%         1067697160   0            0                      100.0%
lonsid_k8s_prd-2021-52                                1     8.6h  peer           index 192.168.10.28  es-node03   172.20.96.95   es-node003  n/a        n/a      165   155             93.9%         165         2008620609  858883303       42.8%         2008620609   0            0                      100.0%
apm-7.16.3-transaction-000001                         0     8.6h  peer           index 192.168.10.26  es-node01   172.20.96.95   es-node003  n/a        n/a      148   122             82.4%         148         26157335871 1312792580      5.0%          26157335871  0            0                      100.0%
apm-7.16.3-transaction-000001                         0     8.6h  peer           index 192.168.10.26  es-node01   172.20.134.162 es-node002  n/a        n/a      148   119             80.4%         148         26157335871 748435112       2.9%          26157335871  0            0                      100.0%
apm-7.16.3-transaction-000002                         0     7.6h  peer           index 192.168.10.28  es-node03   172.20.134.162 es-node002  n/a        n/a      127   115             90.6%         127         7960402426  798748500       10.0%         7960402426   0            0                      100.0%
apm-7.14.0-span-000002                                0     8.6h  peer           index 192.168.10.26  es-node01   172.20.134.162 es-node002  n/a        n/a      253   195             77.1%         253         95785375602 713756422       0.7%          95785375602  0            0                      100.0%
apm-7.14.0-span-000003                                0     8.6h  peer           index 192.168.10.26  es-node01   172.20.96.95   es-node003  n/a        n/a      232   182             78.4%         232         95436963747 1287328824      1.3%          95436963747  0            0                      100.0%
apm-7.14.0-span-000003                                0     8.6h  peer           index 192.168.10.26  es-node01   172.20.134.162 es-node002  n/a        n/a      232   176             75.9%         232         95436963747 704465686       0.7%          95436963747  0            0                      100.0%

This file has been truncated. show original

system · April 12, 2022, 10:56am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.