# Backup indices and data stored in ELK

**URL:** <https://discuss.elastic.co/t/backup-indices-and-data-stored-in-elk/313003>\
**Category:** Elasticsearch\
**Tags:** snapshot-and-restore\
**Created:** [August 26, 2022, 9:23am UTC](https://discuss.elastic.co/t/backup-indices-and-data-stored-in-elk/313003 "2022-08-26T09:23:09Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Clonky](https://avatars.discourse-cdn.com/v4/letter/c/22d042/32.png) [@Clonky](https://discuss.elastic.co/u/Clonky)\
**Post date:** [August 26, 2022, 9:23am UTC](https://discuss.elastic.co/t/backup-indices-and-data-stored-in-elk/313003/1 "2022-08-26T09:23:09Z")

</div>

HI guys, I am rather new to ELK (7.16). So far I have registered a repository and can create now snapshots. I have used the following API command:

```auto
PUT /_snapshot/BackUpELKData/snapshot_2?wait_for_completion=true
{

"indices": "-.ds-ilm-history-5-2022.08.23-000001",

"ignore_unavailable": true,

"include_global_state": false

}

```

The index I excluded caused problems when restoring from the snapshot.  
My goal however is to create a backup of the used indices and the data files.

When I created the snapshot with some indices and data available for Discovery and Dashboards and deleted the indices after creation, I tried to restore them with the snapshot.  
The process of restoring was working with no problems. I received no errors.  
The indices however where still missing so I could not access the data. Neither in Discovery nor in Dashboards.  
I have read [here](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/snapshot-restore.html) about snapshots and I am not sure if I have done everything correctly.  
Can someone help me and explain how I can create a full backup of all data and indices which I can restore at any point?

All the best,  
Clonky

---

<div class="post-metadata">

**Author:** ![Clonky](https://avatars.discourse-cdn.com/v4/letter/c/22d042/32.png) [@Clonky](https://discuss.elastic.co/u/Clonky)\
**Post date:** [August 26, 2022, 11:03am UTC](https://discuss.elastic.co/t/backup-indices-and-data-stored-in-elk/313003/2 "2022-08-26T11:03:27Z")

</div>

If I want to restore the snapshot, I use the API command

```auto
POST /_snapshot/BackUpELKData/snapshot/_restore
{
  "indices": "pq_reports"
}

```

However, I get the error message

```auto
{
  "error" : {
    "root_cause" : [
      {
        "type" : "index_not_found_exception",
        "reason" : "no such index [pq_reports]",
        "index_uuid" : "_na_",
        "index" : "pq_reports"
      }
    ],
    "type" : "index_not_found_exception",
    "reason" : "no such index [pq_reports]",
    "index_uuid" : "_na_",
    "index" : "pq_reports"
  },
  "status" : 404
}

```

This message comes when I want to restore from a snapshot when the indices were still open and when I closed the indices before creating the snapshot

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 29, 2022, 3:48am UTC](https://discuss.elastic.co/t/backup-indices-and-data-stored-in-elk/313003/3 "2022-08-29T03:48:48Z")

</div>

What is the output from [Get snapshot API | Elasticsearch Guide [8.4] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/8.4/get-snapshot-api.html)?

---

<div class="post-metadata">

**Author:** ![Clonky](https://avatars.discourse-cdn.com/v4/letter/c/22d042/32.png) [@Clonky](https://discuss.elastic.co/u/Clonky)\
**Post date:** [August 29, 2022, 6:25am UTC](https://discuss.elastic.co/t/backup-indices-and-data-stored-in-elk/313003/4 "2022-08-29T06:25:52Z")

</div>

Good Morning, the output of

```auto
GET /_snapshot/my_repository/my_snapshot

```

is

```auto
{
  "error" : {
    "root_cause" : [
      {
        "type" : "repository_missing_exception",
        "reason" : "[my_repository] missing"
      }
    ],
    "type" : "repository_missing_exception",
    "reason" : "[my_repository] missing"
  },
  "status" : 404
}

```

The output of the backup API command is

```auto
{
  "snapshot" : {
    "snapshot" : "snapshot_1",
    "uuid" : "eoq0PodYTnGz94f7GfMgXw",
    "repository" : "BackUpELKData",
    "version_id" : 7160399,
    "version" : "7.16.3",
    "indices" : [],
    "data_streams" : [],
    "include_global_state" : false,
    "state" : "SUCCESS",
    "start_time" : "2022-08-29T06:25:25.585Z",
    "start_time_in_millis" : 1661754325585,
    "end_time" : "2022-08-29T06:25:25.585Z",
    "end_time_in_millis" : 1661754325585,
    "duration_in_millis" : 0,
    "failures" : [],
    "shards" : {
      "total" : 0,
      "failed" : 0,
      "successful" : 0
    },
    "feature_states" : []
  }
}

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [August 29, 2022, 6:39am UTC](https://discuss.elastic.co/t/backup-indices-and-data-stored-in-elk/313003/5 "2022-08-29T06:39:51Z")

</div>

> [@Clonky](#):
>
> ....  
> ` "indices" : [],`  
> ...

You are not naming any indices to be added to the snapshot.

Please show the full snapshot API call... That provided that result.

---

<div class="post-metadata">

**Author:** ![Clonky](https://avatars.discourse-cdn.com/v4/letter/c/22d042/32.png) [@Clonky](https://discuss.elastic.co/u/Clonky)\
**Post date:** [August 29, 2022, 6:52am UTC](https://discuss.elastic.co/t/backup-indices-and-data-stored-in-elk/313003/6 "2022-08-29T06:52:18Z")

</div>

Hello, the whole API call that provided the output is

```auto
PUT /_snapshot/BackUpELKData/snapshot_1?wait_for_completion=true
{

"indices": "-.ds-ilm-history-5-2022.08.23-000001",

"ignore_unavailable": true,

"include_global_state": false

}

```

I thought that with this I just excluded the one indice but included all the other ones. Especially the ones I created for my data

All the best

---

<div class="post-metadata">

**Author:** ![Clonky](https://avatars.discourse-cdn.com/v4/letter/c/22d042/32.png) [@Clonky](https://discuss.elastic.co/u/Clonky)\
**Post date:** [August 29, 2022, 12:48pm UTC](https://discuss.elastic.co/t/backup-indices-and-data-stored-in-elk/313003/7 "2022-08-29T12:48:25Z")

</div>

After some more research with the different APIs, I found the command

```auto
PUT /_snapshot/BackUpELKData/my_snapshot

```

which created a snapshot of all the indices. See the result of

```auto
GET /_snapshot/BackUpELKData/my_snapshot

```

```auto
{
  "snapshots" : [
    {
      "snapshot" : "my_snapshot",
      "uuid" : "hUq_mzefT4af4azpFgqQ_Q",
      "repository" : "BackUpELKData",
      "version_id" : 7160399,
      "version" : "7.16.3",
      "indices" : [
        "test-results-abcd",
        "test-results-efgh",
        "ijk_results",
        ".ds-.logs-deprecation.elasticsearch-default-2022.08.23-000001",
        "lmno_logging",
        ".async-search",
        ".kibana_task_manager_7.15.2_001",
        ".ds-.slm-history-5-2022.08.25-000001",
        ".ds-ilm-history-5-2022.08.23-000001",
        "pq_reports",
        ".apm-agent-configuration",
        ".ds-ilm-history-5-2022.08.23-000001xxxx",
        ".kibana-event-log-7.15.2-000001",
        "flashspeed",
        ".apm-custom-link",
        ".kibana_7.15.2_001"
      ],
      "data_streams" : [
        "ilm-history-5",
        ".logs-deprecation.elasticsearch-default",
        ".slm-history-5"
      ],
      "include_global_state" : true,
      "state" : "SUCCESS",
      "start_time" : "2022-08-29T12:45:33.799Z",
      "start_time_in_millis" : 1661777133799,
      "end_time" : "2022-08-29T12:45:34.612Z",
      "end_time_in_millis" : 1661777134612,
      "duration_in_millis" : 813,
      "failures" : [],
      "shards" : {
        "total" : 16,
        "failed" : 0,
        "successful" : 16
      },
      "feature_states" : [
        {
          "feature_name" : "async_search",
          "indices" : [
            ".async-search"
          ]
        },
        {
          "feature_name" : "kibana",
          "indices" : [
            ".apm-custom-link",
            ".apm-agent-configuration",
            ".kibana_7.15.2_001",
            ".kibana_task_manager_7.15.2_001"
          ]
        }
      ]
    }
  ],
  "total" : 1,
  "remaining" : 0
}

```

With this, I can now restore any index via the command

```auto
POST _snapshot/BackUpELKData/my_snapshot/_restore
{
  "indices": "lmno_logging"
}

```

But how can I exclude indices from the backup? Lets say, I do not want to backup the index "lmno\_logging". How should I change the command from above which created the snapshot?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [August 29, 2022, 2:45pm UTC](https://discuss.elastic.co/t/backup-indices-and-data-stored-in-elk/313003/8 "2022-08-29T14:45:01Z")

</div>

> [@Clonky](#):
>
> ```auto
> PUT /_snapshot/BackUpELKData/snapshot_1?wait_for_completion=true
> {
> 
> "indices": "-.ds-ilm-history-5-2022.08.23-000001",
> 
> ```

No that just says ingore that one... and give no other instructions... so no indices are backed up I think you were trying to

`"indices": "*,-.ds-ilm-history-5-2022.08.23-000001",`

Which would say all but that index.... Once you put an index in there you need to be precise.

---

<div class="post-metadata">

**Author:** ![Clonky](https://avatars.discourse-cdn.com/v4/letter/c/22d042/32.png) [@Clonky](https://discuss.elastic.co/u/Clonky)\
**Post date:** [August 29, 2022, 2:58pm UTC](https://discuss.elastic.co/t/backup-indices-and-data-stored-in-elk/313003/9 "2022-08-29T14:58:03Z")

</div>

Thanks a lot! The snipped is working now.

Thanks for your help!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 26, 2022, 2:58pm UTC](https://discuss.elastic.co/t/backup-indices-and-data-stored-in-elk/313003/10 "2022-09-26T14:58:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.