S3 Snapshot backing up newly made indexes

Elastic Stack Elasticsearch
1

Hello,
I am having issues backing up a new index to my S3 bucket

About my setup
I am using Elk 6.8.0 with a new index being made each day with the following format
LogStash-(Year, month day)

I am also using the ElasticSearch S3 SnapShot plugin.

I was able to back up some of my indexes to an S3 bucket from running the following command, and it worked!!!!!.

PUT _snapshot/my_s3_repository/backup
{
  "type": "s3",
  "settings": {
    "bucket": "ts-threat-indexes",
    "compress": true,
    "region": "us-east-2b"
  }
}
logstash-2019.06.07
logstash-2019.06.08
logstash-2019.06.09
logstash-2019.06.10
logstash-2019.06.11
logstash-2019.06.12
logstash-2019.06.13
logstash-2019.06.14
logstash-2019.06.15
logstash-2019.06.16
logstash-2019.06.17
logstash-2019.06.18
logstash-2019.06.19
logstash-2019.06.21
logstash-2019.06.22
logstash-2019.06.23
logstash-2019.06.24
logstash-2019.06.25
logstash-2019.06.26
logstash-2019.06.27

My Issue.

A new index was made and I want to back it up

logstash-2019.06.30

I ran the following command

PUT _snapshot/my_s3_repository/backup/
{
  "indices": "logstash-2019.06.30",
  "include_global_state": false
  }

But got an error :frowning:

{
  "error": {
    "root_cause": [
      {
        "type": "invalid_snapshot_name_exception",
        "reason": "[my_s3_repository:backup] Invalid snapshot name [backup], snapshot with the same name already exists"
      }
    ],
    "type": "invalid_snapshot_name_exception",
    "reason": "[my_s3_repository:backup] Invalid snapshot name [backup], snapshot with the same name already exists"
  },
  "status": 400
}

This all said when I run the code below, it works fine.

PUT _snapshot/my_s3_repository/backup/logstash-2019.06.30/
{
  "indices": "logstash-2019.06.30",
  "include_global_state": false
  }

Do I have to make a new ElasticSearch S3 repo every time I want to take a backup/snapshot of a newly formed index?

@dadoonet Do you have any words of wisdom?

2

How do i add a snapshot to an /backup/ repo?

3

Please don't ping people who are not yet involved in your thread.

4

That should work I guess...

PUT _snapshot/my_s3_repository/backup/snap1
{
  "indices": "logstash-2019.06.30",
  "include_global_state": false
}
PUT _snapshot/my_s3_repository/backup/snap2
{
  "indices": "logstash-2019.07.01",
  "include_global_state": false
}
5

is there any other way to add an index to an exiting snapshot?

6

Take a new snapshot with both indices and remove the old snapshot.
If no changes happened in the first Index, it will just copy the data from the new index.