How to use shared drive location for path.repo between linux and Windows machines in cluster?

Elastic Stack Elasticsearch
1

I am using a cluster of about 3-4 nodes in which Master is always a Windows machine.
How can I use linux node in this setup with the shared drive in Windows to take the snapshot?

Is there any documentation for this?

2

I don't think there's a way to do this with a shared-filesystem repository. The path to the repository has to be the same across all machines, and the syntax of paths is different on Windows vs on Linux. Perhaps there's some kind of compatibility layer that might work but I don't really know.

More generally I don't think mixed-OS clusters are a supported configuration. I'm pretty sure it's not something that is tested. It's probably best to move onto a single OS if you can.

It is also possible that you could use the repository-s3 plugin together with Minio to take snapshots in this cluster instead of using a shared-filesystem repository.

1 Like
3

Is there any other way to take a snapshot of a mixed-OS clusters? Also, is

Shared-filesystem repository is the only repository we can use for snapshot?

4

There are plugins available that allow you to take snapshots to S3, Google cloud storage, Azure cloud storage and HDFS.

5

Yes, I gave you one such suggestion already:

1 Like
6

Is there a way to take snapshot of a mixed Os cluster which is not connected to the Internet?

7

Yes, you can use the repository-s3 plugin together with Minio.

8

I have found out a workaround for taking snapshot in a mixed OS Cluster since I can't move to a single OS cluster. Please check and tell if it will create any issue.

  • CreateRepo with verify as false in a repo that is accessible from the Master node

  • Move the Index to be snapshot to the Master node using index.routing.allocation.require._name setting.

  • Once all the shards have moved to Master Node, give CreateSnapshot command, Master node will create a snapshot in the created repository with all the shards.

9

Or, you know, you could use the repository-s3 plugin together with Minio! :slight_smile:

I do not think your proposal works very reliably. You need to be sure that the primary of the index is on the master, but there's no way to control where the primary is unless you remove all the replicas too. But this means that if the master node fails while you're taking a snapshot then you will have lost this whole index.

You will also be putting a bunch of extra load on your cluster moving data onto the master and then back again each time you want to make a snapshot. You will need to develop quite a complex system to do this extra work reliably, to make sure that you've taken a snapshot of everything that you wanted. You will have to account for all the possible failure modes (master re-election, node failure, etc. etc.) and be sure to correctly clean up afterwards.

1 Like
10

You need to be sure that the primary of the index is on the master, but there's no way to control where the primary is unless you remove all the replicas too.

I am removing the replicas of the particular index before moving the the shards to the master.

This workaround is still under testing. Will update if I face any issue.

Thank you for the response.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.