ES, KIBANA AND SNAPSHOT cluster with two machines (nodes) - ES version 8.15

Elastic Stack Elasticsearch
1

Hi, i have a cluster with two nodes.
machine one - master - roles - "master", "data", "ml", "remote_cluster_client"
machine two - second - data - roles - "data"

machine tree - kibana - snapshot - this machine don't have ES intalled.

NFS - server at machine tree. folder nfs - "/home/snapshot/sharing/backup"
I've already tried - chmod -R 755 and chmod -R 777.

NFS - cliente at machine one and two.

machine one and two has access to a mount folder and configurated this at
path.repo = ["/home/snapshot/backup"]

The problem is!
When i try to verify a repository at kibana show me.

{
  "name": "ResponseError",
  "message": "repository_verification_exception\n\tRoot causes:\n\t\trepository_verification_exception: [database] [[8TNA1PNTT1G8XBx49oNoug, 'org.elasticsearch.transport.RemoteTransportException: [node-2][192.168.1.137:9300][internal:admin/repository/verify]']]"
}

The error at master node is:

node-1 master

[2024-09-13T20:39:13,907][WARN ][r.suppressed             ] [node-1] path: /_snapshot/database/_verify, params: {repository=database}, status: 500
org.elasticsearch.repositories.RepositoryVerificationException: [database] [[8TNA1PNTT1G8XBx49oNoug, 'org.elasticsearch.transport.RemoteTransportException: [node-2][192.168.1.137:9300][internal:admin/repository/verify]']]
	Suppressed: org.elasticsearch.transport.RemoteTransportException: [node-2][192.168.1.137:9300][internal:admin/repository/verify]
	Caused by: org.elasticsearch.repositories.RepositoryVerificationException: [database] store location [/home/snapshot/backup] is not accessible on the node [{node-2}{8TNA1PNTT1G8XBx49oNoug}{ZjcoBOafRZGto9QcQzGfyA}{node-2}{192.168.1.137}{192.168.1.137:9300}{d}{8.15.1}{7000099-8512000}{ml.config_version=12.0.0, xpack.installed=true, transform.config_version=10.0.0}]
	Caused by: java.nio.file.AccessDeniedException: /home/snapshot/backup/tests-5BCRWDWASYCbXTwLFPgLSw/data-8TNA1PNTT1G8XBx49oNoug.dat

node-2

[2024-09-13T16:39:14,947][WARN ][o.e.r.V.TransportAction  ] [node-2] [database] failed to verify repository
org.elasticsearch.repositories.RepositoryVerificationException: [database] store location [/home/snapshot/backup] is not accessible on the node [{node-2}{8TNA1PNTT1G8XBx49oNoug}{ZjcoBOafRZGto9QcQzGfyA}{node-2}{192.168.1.137}{192.168.1.137:9300}{d}{8.15.1}{7000099-8512000}{ml.config_version=12.0.0, xpack.installed=true, transform.config_version=10.0.0}]
Caused by: java.nio.file.AccessDeniedException: /home/snapshot/backup/tests-XJA0yMcSQPW8TxFjZtjmrQ/data-8TNA1PNTT1G8XBx49oNoug.dat

When execute this - /_snapshot/xxxx/_verify?pretty

{
    "error": {
        "root_cause": [
            {
                "type": "repository_verification_exception",
                "reason": "[database] [[8TNA1PNTT1G8XBx49oNoug, 'org.elasticsearch.transport.RemoteTransportException: [node-2][192.168.1.137:9300][internal:admin/repository/verify]']]",
                "suppressed": [
                    {
                        "type": "repository_verification_exception",
                        "reason": "[database] store location [/home/snapshot/backup] is not accessible on the node [{node-2}{8TNA1PNTT1G8XBx49oNoug}{ZjcoBOafRZGto9QcQzGfyA}{node-2}{192.168.1.137}{192.168.1.137:9300}{d}{8.15.1}{7000099-8512000}{ml.config_version=12.0.0, xpack.installed=true, transform.config_version=10.0.0}]"
                    }
                ]
            }
        ],
        "type": "repository_verification_exception",
        "reason": "[database] [[8TNA1PNTT1G8XBx49oNoug, 'org.elasticsearch.transport.RemoteTransportException: [node-2][192.168.1.137:9300][internal:admin/repository/verify]']]",
        "suppressed": [
            {
                "type": "repository_verification_exception",
                "reason": "[database] store location [/home/snapshot/backup] is not accessible on the node [{node-2}{8TNA1PNTT1G8XBx49oNoug}{ZjcoBOafRZGto9QcQzGfyA}{node-2}{192.168.1.137}{192.168.1.137:9300}{d}{8.15.1}{7000099-8512000}{ml.config_version=12.0.0, xpack.installed=true, transform.config_version=10.0.0}]",
                "caused_by": {
                    "type": "access_denied_exception",
                    "reason": "/home/snapshot/backup/tests-kNMEK4poR9etxHFMv3_WIA/data-8TNA1PNTT1G8XBx49oNoug.dat"
                }
            }
        ]
    },
    "status": 500
}

node-1 master - create a folder and files with this user
drwxr-xr-x 2 elasticsearch elasticsearch 4096 Sep 13 15:46

node-2 - create a folder and files with this user
drwxr-xr-x 2 usbmux uuidd 4096 Sep 13 15:46

Please can someone help with this? Thanks

2

Hi,

I found the solution for this case!

Type ID on eacth node machine and get a number from root user.

uid=0(root) gid=0(root) groups=0(root)

after that in your file /etc/exports include this info. anonuid and anongid

/folder client_ip(rw,sync,all_squash,no_subtree_check,anonuid=0,anongid=0)

restart the nfs and that is each.