Hello from Japan
I have a question for all the dear Elastic engineers
I want to use the snapshot function to save a snapshot on one node in a cluster.
*I have configured a cluster of three Elasticsearch machines.
I have introduced Samba to the server to store the snapshot repository.
I am trying to create a shared folder with Samba and store the repository there.
However, many errors are bothering me.
Authentication status = Not connected
{
"name": "ResponseError",
"message": "repository_verification_exception\n\tCaused by:\n\t\texception: failed to create blob container\n\tRoot causes:\n\t\texception: failed to create blob container"
}
Error message when the shared folder source is set as the master.
*Server names and IP addresses are protected for security reasons.
Authentication status = Not connected
{
"name": "ResponseError",
"message": "repository_verification_exception\n\tRoot causes:\n\t\trepository_verification_exception: [repo_test] [[EPknCeSwRISYedwlqybijA, 'org.elasticsearch.transport.RemoteTransportException: [server name][xxx.xxx.xxx.xxx:xxxx][internal:admin/repository/verify]'], [-zN11OFCSPSBnvYOYu5kQg, 'org.elasticsearch.transport.RemoteTransportException: [server name][xxx.xxx.xxx.xxx:xxxx][internal:admin/repository/verify]']]"
}
☆My execution environment is as follows:
OS:Ubuntu 22.04
Samba：4.15.13
Elasticsearch：8.13.4
Kibana：8.13.4
Each elasticsearch.yml file contains the repository settings as follows:
path.repo: /hayato
Similar issues have been reported on the following sites, and I have investigated them but have been unable to resolve them.
I would appreciate any advice on the cause of the error, how to resolve it, and how to obtain a snapshot.
【The results of a survey I conducted on my own.】
☆I suspected a problem with file system permissions, but I'm not sure of the details.
①The original permissions for the shared folder are displayed as drwxrwxrwx root root.
When viewed from other mounted nodes, they are displayed as drwxr-xr-x root root, and cannot be changed with the chmod, chgrp, or chown commands.
This may be why writing is not possible and the snapshot may be failing, but I don't know how to resolve it.
※I have confirmed that I can access the shared folder and write files from my own laptop.
②I checked the following site and found that the UID and GID of each node were different, so I changed them to the same values, but now elasticsearch.service won't start.
Many NFS implementations match accounts across nodes using their numeric user IDs (UIDs) and group IDs (GIDs) rather than their names.
It is possible for Elasticsearch to run under an account with the same name (often ) on each node,
but for these accounts to have different numeric user or group IDs.
If your shared file system uses NFS then ensure that every node is running with the same numeric UID and GID,
or else update your NFS configuration to account for the variance in numeric IDs across nodes.elasticsearch
The Samba configuration file, smb.conf, is written as follows:
[CCR]
path = /hayato
browsable = yes
writable = yes
guest ok = yes
guest only = yes
read only = no
force create mode = 777
force directory mode = 777
I am in a very difficult situation.
Please, fellow Elasticsearch engineers, share your wisdom.
Regards
Thank you
※reference infomation
The following is written in the Samba configuration file, smb.conf.
/hayato is mounted as a shared folder on other nodes.
Filesystem Size Used Avail Use% Mounted on
tmpfs 794M 1.1M 793M 1% /run
/dev/vda1 1.9T 37G 1.9T 2% /
tmpfs 3.9G 28K 3.9G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/vda15 105M 6.1M 99M 6% /boot/efi
tmpfs 794M 4.0K 794M 1% /run/user/1010
tmpfs 794M 4.0K 794M 1% /run/user/1001
//xxx.xxx.xxx.xxx/CCR 1000G 57G 944G 6% /hayato
This isn't really an Elasticsearch problem, at least not now we've narrowed it down to the