Snapshot and restore in a cluster with both Windows and Linux

Elastic Stack Elasticsearch
1

I am stuck in setting up a shared file system for my cluster which has both windows and linux systems.

The official guide seems to assume a linux environment as it mentions that "it is necessary to mount the same shared filesystem to the same location on all master and data nodes".

I have no idea of how to set up a shared file system yet. Even if I could do that, how to configure the location property of the backup is another problem.

2

I'm going to add a related question. I'm trying to restore a snapshot taken in our staging environment, which runs on Linux, to my local ElasticSearch instance running on Windows. I see that when I take a snapshot of an index in my Windows environment, I see two files that I don't see in the snapshot taken from the Linux environment: meta-[snapshot name] and snapshot-[snapshot name].

I've been able to "successfully" restore a snapshot from Linux onto Windows, but looking at the index, there is no data in it. The response from the API call indicated a success, but I question this.

Is this even possible to restore a snapshot from Linux to Windows? The difference between the files I see in the snapshot make me question this. Thanks for reading.

3

I had some progress today. I installed a semba server in a linux machine. I then verified that I could manually create and edit files in that shared folder from the windows machine. Then I proceed to configure my windows elasticsearch.yml as follows:

path.repo: ["\\\\10.162.59.164\\es_ubuntu"]

Then I tried:

POST _snapshot/fht360_backup
{
  "type": "fs",
  "settings":{
    "location": "./fht360_backup"
  }
}

But then I experienced an error:

{
    "error": "RepositoryException[[fht360_backup] failed to create repository]; nested: CreationException[Guice creation errors:\r\n\r\n1) Error injecting constructor, org.elasticsearch.common.blobstore.BlobStoreException: Failed to create directory at [\\\\10.162.59.163\\es_ubuntu\\fht360_backup]\r\n  at org.elasticsearch.repositories.fs.FsRepository.<init>(Unknown Source)\r\n  while locating org.elasticsearch.repositories.fs.FsRepository\r\n  while locating org.elasticsearch.repositories.Repository\r\n\r\n1 error]; nested: BlobStoreException[Failed to create directory at [\\\\10.162.59.163\\es_ubuntu\\fht360_backup]]; ",
    "status": 500
}

To verify that I can programmatically write to that network shared folder, I even wrote a small program in C# and it can write without problems.

4

You have a backtick instead of a figure one in the UNC path. Your elasticsearch.yml setting should look like this:

path.repo: ["\\\10.162.59.164\\es_ubuntu"]

But what are you trying to do? Don't you need the same repository path on all machines in the cluster? While what you've done so far should make it possible to access a Linux file system from Windows, the UNC path won't work on Linux.

1 Like
5

Sorry, it is an edit error in my previous reply. (Don't know why two backlashes followed by a 1 is shown as a backslash and a backtick in markdown. I didn't spot that earlier). Anyway, I am taking screenshots instead. The one in my elasticsearch.yml in the windows machine doesn't have backticks, which is:

The setting in the linux machine was:

I made /.elasticsearch_backup/ a shared folder via samba, so it is seen by the windows machine as:

Note I manually added a testing.txt to show that I have access to that folder. Prove that I could see the same file in the linux box:

Pasted image735×118 41 KB

Then I was thinking that I can use a relative path in my snapshot setting's location property. So each machine will actually write to a different repo path but is actually the same networked directory.

Did you suggest that this won't work and that path.repo should be identical in all elasticsearch.yml files? If that is the case, my dilemma then is that I can only specify path in windows using the two-backslash networked file path syntax [aka UNC path] while in linux I can only specify a path in the unix file path syntax (be it mounted or directly accessed). I cannot make both happy at the same time.