I didn't understand the Elastic Snapshot properly Please help

Elastic Stack Elasticsearch
1

My configuration is to run each elastic on 3 different servers.
Elastic that works as a master node (also a data node) on the 1st server.
Elastic No. 2 and No. 3 operate as data nodes.

I know that I have to specify the same path in all three Elastic configuration files by searching.

However, I have no idea how to write the path.repo.

After configuring Elastic on 3 different servers like this, please do not know how to set path.repo to set up snapshot..

2
  1. You set the Path Repo in the .yml
    path.repo: ["\\\\MY_SERVER\\Snapshots"]
  2. You register that repo
PUT /_snapshot/RepoName
{
    "type": "fs",
    "settings": {
        "location": "\\\\MY_SERVER\\Snapshots\\Repo",
        "compress": true
    }
}
  1. Take the snapshot
    PUT /_snapshot/RepoName/snapshot_1?wait_for_completion=true

Note: This apporach uses windows paths which have to be escaped with . Thats why it has \. Its different in linux.
Find more information here
https://www.elastic.co/guide/en/elasticsearch/reference/7.x/snapshot-restore.html

3

Thank you for your help. I have one question.

Is it okay if the path.repo is not the same in the three elastics?

If so, do i need to combine the three when recovering?

4

A snapshot contains all indices from a cluster (as far as i know). If you enter path.repo on node1 and you use this node for the snapshots you should be finde. Why do you want to enter different paths on different nodes?
To be sure that your snapshot system works:

  1. Set Path.repo on Node1.
  2. Take a snapshot using this Node1.
  3. Set up a new node outside the cluster and try to restore the snapshot.
    If all goes according to plan you will have all the indices from the cluster. Even if they are not on Node1.
    If this works you are all set.
5

Thank you very much for your help. However, my problem has not been solved.

Does it mean to set path.repo on node 1, does it mean that it is not necessary to set on node 2 and node 3?

If you set path.repo on node 1 only and issue a snapshot command, the phrase "failed to verify repository" appears on node 2 and node 3.

I wonder. Shouldn't all nodes look at a single path when specifying a snapshot?.

If not, how can you take a snapshot of the data that is divided into elastics?

Am I misunderstanding?

I have split the Elastic on different ip servers in the Linux environment.

If you need to look at one place as the path of the snapshot, how do you write path.repo on the remaining nodes 2 and 3?

(in elasticsearch-node1.yml)
path.repo: [/home/elasticsearch-node1]

(in elasticsearch-node2,3.yml)
path.repo: [How should I write this part?]

6

@111349

IMHO don't set different values. More importantly why do you want to set different values?

https://www.elastic.co/guide/en/elasticsearch/reference/7.x/snapshots-register-repository.html#snapshots-filesystem-repository

  1. The location path used while registering a shared filesystem repository must resolve to a single directory location (not just path string value) from every master and data node.
  2. One of the directories in the location path must be specified in repo.path.

Let's say you mount a shared directory on 3 ES nodes as /mnt/backup.

You can specify path.repo on 3 nodes as

/mnt/backup/foo/bar/baz
/mnt/backup/foo/bar
/mnt/backup/foo

and repository location as /mnt/backup/foo/bar/baz. Configuration meets both criteria. Your snapshots will work.

However this configuration will not work if use relative path as repository location for ex. my_backup. The first node will resolve the relative path as /mnt/backup/foo/bar/baz/my_backup, second node will resolve as /mnt/backup/foo/bar/my_backup and third node as /mnt/backup/foo/my_backup. Your register repository call will fail.

7

I want to set path.repo to a common directory.

How can I have a common directory on different Linux servers?

Is there such a function in Elastic? Or should I mount it?

9

The repository you use have to be shared storage and accessible from ALL nodes in the cluster. Elasticsearch will test this by having one node write a file and have another one read it. The path has to be set up on all nodes and has to be the same across the board.

10

First of all, thank you so much for helping me with the difficulties I have had for a long time

First, I set path.repo on node1 only
then taking a snapshot. But what I got is an error.
The error came out like this
RemoteTransportException[[node2][node2_ip][internal:admin/repository/verify]]; nested: RepositoryMissingException[[mybackup] missing];
Same for node 3.

So I thought I did that by not setting path.repo on nodes 2 and 3.
However, it is amazing to say that you only need to set node1.
So what am I missing now? Confusing

11

Thank you for your reply

So I'm running Elastic on different Linux servers now, does that mean I need to mount it?

Does it mean that you can set the path.repo of node 2 and 3 to the directory mounted on node 1?

12

Typically you create an NFS volume and mount this on all the nodes in the same path.

1 Like
13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.