I am trying to ALL restore user created indices. I don't want to restore the .geoip_databases, .security, .ds-.logs-deprecation.elasticsearch-default* and so on.
I tried this :-
{
"indices": "*",
"include_global_state": false
}
but I get this error :-
"type": "snapshot_restore_exception",
"reason": "[my_repo:test-snapshot-2/yWuONkjGTIuieFAe7dWeYQ] cannot restore index [.ds-.logs-deprecation.elasticsearch-default-2023.02.01-000001] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name"
Hi @x00m,
maybe would be easier to just add the list of the indices you want to restore instead of using * wildcard?
Is the list of indices too large?
you could try something like
I had to rollover the indices from .ds-logs and ds-ilm to remove the ones that were on the backup, I couldn't exclude for some reason. I then deleted the ds indices from the cluster that were in the backup and run the _restore with the following
POST _snapshot/backup/daily-snap-2023.02.10-y87yklpfra2eh24jar81fa/_restore
{
"indices": "*,-.*,-.apm*,-.geoip*,-.tasks,-.kibana*,-.ds*",
"include_global_state": false
}
Although the ds indices were removed I just kept it in case, I tried with a docker service without authentication, so maybe there are more hiden indices to exclude.
The only ones that gave me problems were the ones starting with ds
Hi @x00m , sure. ds-.logs- indices that your snapshot wants to overwrite are based on ILM templates and they can be rotated.
As in your new cluster you had some of these system indices matching it's name, it couldn't overwrite, so what I did is to rollover the index that matched on the snapshot and the cluster so I could delete from the cluster the one coming from the snapshot.
# check the aliases associated for the rollover
GET .ds-ilm-history-5-2023.02.10-000001
# rollover the index
POST ilm-history-5/_rollover
# delete old index which is also in snapshots
DELETE .ds-ilm-history-5-2023.02.10-000001
# same process for .ds indices
GET .ds-.logs-deprecation.elasticsearch-default-2023.02.10-000001
POST .logs-deprecation.elasticsearch-default/_rollover
DELETE .ds-.logs-deprecation.elasticsearch-default-2023.02.10-000001
Now I don't have the same .ds-logs in snaphost and cluster so I can perform the snapshot recovery with
POST _snapshot/backup/daily-snap-2023.02.10-y87yklpfra2eh24jar81fa/_restore
{
"indices": "*,-.*,-.apm*,-.geoip*,-.tasks,-.kibana*,-.ds*",
"include_global_state": false
}
It makes it fine.
These files are logs generated by elastic and some internal work, I don't think they're much important when restoring backups, maybe you could just remove them, but this is a cleaner procedure.
The rest of internal indices shouldn't give any errors.
Hope this helps.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.