# Restore all User created Indices, exclude the indices starting with

**URL:** <https://discuss.elastic.co/t/restore-all-user-created-indices-exclude-the-indices-starting-with/324428>\
**Category:** Elasticsearch\
**Tags:** snapshot-and-restore\
**Created:** [February 1, 2023, 1:29pm UTC](https://discuss.elastic.co/t/restore-all-user-created-indices-exclude-the-indices-starting-with/324428 "2023-02-01T13:29:17Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![x00m](https://avatars.discourse-cdn.com/v4/letter/x/90db22/32.png) [@x00m](https://discuss.elastic.co/u/x00m)\
**Post date:** [February 1, 2023, 1:29pm UTC](https://discuss.elastic.co/t/restore-all-user-created-indices-exclude-the-indices-starting-with/324428/1 "2023-02-01T13:29:17Z")

</div>

I am trying to ALL restore user created indices. I don't want to restore the .geoip\_databases, .security, .ds-.logs-deprecation.elasticsearch-default\* and so on.

I tried this :-

> {  
> "indices": "\*",  
> "include\_global\_state": false  
> }

but I get this error :-

> "type": "snapshot\_restore\_exception",  
> "reason": "[my\_repo:test-snapshot-2/yWuONkjGTIuieFAe7dWeYQ] cannot restore index [.ds-.logs-deprecation.elasticsearch-default-2023.02.01-000001] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name"

So, I tried this :-

> {  
> "indices": "-.ds-.logs-deprecation.elasticsearch-default\*, \*",  
> "include\_global\_state": false  
> }

But then I get the same error, but stating that an index with name **.geoip\_databases** already exists, and then .security-7

So this is what i tried:

> {  
> "indices": "-.ds-.logs-deprecation.elasticsearch-default\*, -.geoip\_databases, -.security-7, \*",  
> "include\_global\_state": false  
> }

This then returns 404 with error :-

> {  
> "error": {  
> "root\_cause": [  
> {  
> "type": "index\_not\_found\_exception",  
> "reason": "no such index [-.geoip\_databases]",  
> "index\_uuid": "_na_",  
> "index": " -.geoip\_databases"  
> }  
> ],  
> "type": "index\_not\_found\_exception",  
> "reason": "no such index [-.geoip\_databases]",  
> "index\_uuid": "_na_",  
> "index": " -.geoip\_databases"  
> },  
> "status": 404  
> }

How to solve this?

Eck 2.3.0, elasticsearch 7.17.8

---

<div class="post-metadata">

**Author:** ![jacinto\_calvo\_sintes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jacinto_calvo_sintes/32/87338_2.png) [@jacinto\_calvo\_sintes](https://discuss.elastic.co/u/jacinto_calvo_sintes)\
**Post date:** [February 7, 2023, 4:07pm UTC](https://discuss.elastic.co/t/restore-all-user-created-indices-exclude-the-indices-starting-with/324428/2 "2023-02-07T16:07:12Z")

</div>

Hi @x00m,  
maybe would be easier to just add the list of the indices you want to restore instead of using \* wildcard?  
Is the list of indices too large?  
you could try something like

```auto
{
"indices": ["logs-*", "index-1", "index-2"] ,
"include_global_state": false
}

```

Hope this helps.

---

<div class="post-metadata">

**Author:** ![x00m](https://avatars.discourse-cdn.com/v4/letter/x/90db22/32.png) [@x00m](https://discuss.elastic.co/u/x00m)\
**Post date:** [February 10, 2023, 1:06pm UTC](https://discuss.elastic.co/t/restore-all-user-created-indices-exclude-the-indices-starting-with/324428/3 "2023-02-10T13:06:02Z")

</div>

The list of indices is large (few thousands). I just want to restore all indices except the ones starting with a dot

---

<div class="post-metadata">

**Author:** ![jacinto\_calvo\_sintes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jacinto_calvo_sintes/32/87338_2.png) [@jacinto\_calvo\_sintes](https://discuss.elastic.co/u/jacinto_calvo_sintes)\
**Post date:** [February 13, 2023, 5:01pm UTC](https://discuss.elastic.co/t/restore-all-user-created-indices-exclude-the-indices-starting-with/324428/4 "2023-02-13T17:01:24Z")

</div>

Hi,  
I did some tests and as you said, the `_restore` option using both wildcards `*,-.*` doesn't seem to work. At least the tests I did had the same behavior as your case. I found that open issue that might have relation [snapshot with exclude expression `-` does not work if you specify explicit index name · Issue #83435 · elastic/elasticsearch · GitHub](https://github.com/elastic/elasticsearch/issues/83435)  
I managed to restore the snapshot the way you want doing some steps.

I had to rollover the indices from `.ds-logs` and `ds-ilm` to remove the ones that were on the backup, I couldn't exclude for some reason. I then deleted the `ds` indices from the cluster that were in the backup and run the \_restore with the following

```auto
POST _snapshot/backup/daily-snap-2023.02.10-y87yklpfra2eh24jar81fa/_restore
{
  "indices": "*,-.*,-.apm*,-.geoip*,-.tasks,-.kibana*,-.ds*",
  "include_global_state": false 
}

```

Although the `ds` indices were removed I just kept it in case, I tried with a docker service without authentication, so maybe there are more hiden indices to exclude.  
The only ones that gave me problems were the ones starting with `ds`

---

<div class="post-metadata">

**Author:** ![x00m](https://avatars.discourse-cdn.com/v4/letter/x/90db22/32.png) [@x00m](https://discuss.elastic.co/u/x00m)\
**Post date:** [February 15, 2023, 9:08am UTC](https://discuss.elastic.co/t/restore-all-user-created-indices-exclude-the-indices-starting-with/324428/5 "2023-02-15T09:08:56Z")

</div>

> [@jacinto\_calvo\_sintes](#):
>
> I had to rollover the indices from `.ds-logs` and `ds-ilm` to remove the ones that were on the backup

Can you please shed some more light on what you mean by this?

---

<div class="post-metadata">

**Author:** ![jacinto\_calvo\_sintes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jacinto_calvo_sintes/32/87338_2.png) [@jacinto\_calvo\_sintes](https://discuss.elastic.co/u/jacinto_calvo_sintes)\
**Post date:** [February 16, 2023, 6:01pm UTC](https://discuss.elastic.co/t/restore-all-user-created-indices-exclude-the-indices-starting-with/324428/6 "2023-02-16T18:01:44Z")

</div>

Hi @x00m , sure.  
`ds-.logs-` indices that your snapshot wants to overwrite are based on ILM templates and they can be rotated.  
As in your new cluster you had some of these system indices matching it's name, it couldn't overwrite, so what I did is to rollover the index that matched on the `snapshot` and the `cluster` so I could delete from the cluster the one coming from the `snapshot`.

```auto
# check the aliases associated for the rollover
GET .ds-ilm-history-5-2023.02.10-000001 
# rollover the index
POST ilm-history-5/_rollover
# delete old index which is also in snapshots
DELETE .ds-ilm-history-5-2023.02.10-000001 

# same process for .ds indices
GET .ds-.logs-deprecation.elasticsearch-default-2023.02.10-000001

POST .logs-deprecation.elasticsearch-default/_rollover

DELETE .ds-.logs-deprecation.elasticsearch-default-2023.02.10-000001

```

Now I don't have the same `.ds-logs` in `snaphost` and `cluster` so I can perform the `snapshot recovery` with

```auto
POST _snapshot/backup/daily-snap-2023.02.10-y87yklpfra2eh24jar81fa/_restore
{
  "indices": "*,-.*,-.apm*,-.geoip*,-.tasks,-.kibana*,-.ds*",
  "include_global_state": false 
}

```

It makes it fine.  
These files are logs generated by elastic and some internal work, I don't think they're much important when restoring backups, maybe you could just remove them, but this is a cleaner procedure.  
The rest of internal indices shouldn't give any errors.  
Hope this helps.

---

<div class="post-metadata">

**Author:** ![x00m](https://avatars.discourse-cdn.com/v4/letter/x/90db22/32.png) [@x00m](https://discuss.elastic.co/u/x00m)\
**Post date:** [February 17, 2023, 5:16am UTC](https://discuss.elastic.co/t/restore-all-user-created-indices-exclude-the-indices-starting-with/324428/7 "2023-02-17T05:16:12Z")

</div>

Thank you so much! I'll try this out!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 17, 2023, 5:16am UTC](https://discuss.elastic.co/t/restore-all-user-created-indices-exclude-the-indices-starting-with/324428/8 "2023-03-17T05:16:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.