Elaticsearch not recognising path repo

Elastic Stack Elasticsearch
1

Hi,

My elasticsearch applications are running on docker. i have mounted the path.repo but i am getting following error:

{"error":"RepositoryException[[my_backup] failed to create repository]; nested: CreationException[Guice creation errors:\n\n1) Error injecting constructor, org.elasticsearch.repositories.RepositoryException: [my_backup] location [/mnt/es-snaps] doesn't match any of the locations specified by path.repo\n at org.elasticsearch.repositories.fs.FsRepository.(Unknown Source)\n while locating org.elasticsearch.repositories.fs.FsRepository\n while locating org.elasticsearch.repositories.Repository\n\n1 error]; nested: RepositoryException[[my_backup] location [/mnt/es-snaps] doesn't match any of the locations specified by path.repo]; ","status":500}

Can you please help me out.
Thanks in advance.

Thank you

2

What do you have in your elasticsearch.yml file?

3

Hi David,

path.repo: ["/mnt/es-snaps"]

Following is the output for

curl "localhost:9200/_nodes/settings?pretty"

{
  "cluster_name" : "elasticsearch",
  "nodes" : {
    "BKJL7TS_Q5KC3dm8Xep2UA" : {
      "name" : "es-node-2",
      "transport_address" : "inet[/10.0.1.21:9300]",
      "host" : "7459d736d6e1",
      "ip" : "10.0.1.21",
      "version" : "1.7.5",
      "build" : "00f95f4",
      "http_address" : "inet[/10.0.1.21:9200]",
      "settings" : {
        "path" : {
          "logs" : "/usr/share/elasticsearch/logs",
          "home" : "/usr/share/elasticsearch",
          "repo" : "/usr/share/elasticsearch/snapshots"
        },
        "cluster" : {
          "name" : "elasticsearch"
        },
        "node" : {
          "name" : "es-node-2"
        },
        "discovery" : {
          "zen" : {
            "ping" : {
              "unicast" : {
                "hosts" : "es-node-1"
              }
            }
          }
        },
        "name" : "es-node-2",
        "client" : {
          "type" : "node"
        },
        "foreground" : "yes",
        "config" : {
          "ignore_system_properties" : "true"
        },
        "script" : {
          "inline" : "on",
          "update" : "on",
          "groovy" : "on"
        }
      }
    },
    "-Nb8FWW_RPWL07yQCcD6QQ" : {
      "name" : "es-node-3",
      "transport_address" : "inet[/10.0.1.22:9300]",
      "host" : "8b0eda835741",
      "ip" : "10.0.1.22",
      "version" : "1.7.5",
      "build" : "00f95f4",
      "http_address" : "inet[/10.0.1.22:9200]",
      "settings" : {
        "path" : {
          "logs" : "/usr/share/elasticsearch/logs",
          "home" : "/usr/share/elasticsearch",
          "repo" : "/usr/share/elasticsearch/snapshots"
        },
        "cluster" : {
          "name" : "elasticsearch"
        },
        "node" : {
          "name" : "es-node-3"
        },
        "discovery" : {
          "zen" : {
            "ping" : {
              "unicast" : {
                "hosts" : "es-node-1"
              }
            }
          }
        },
        "name" : "es-node-3",
        "client" : {
          "type" : "node"
        },
        "foreground" : "yes",
        "config" : {
          "ignore_system_properties" : "true"
        },
        "script" : {
          "inline" : "on",
          "update" : "on",
          "groovy" : "on"
        }
      }
    },
    "n5LJhtymRo2DL5I-FpTsag" : {
      "name" : "es-node-1",
      "transport_address" : "inet[/10.0.1.20:9300]",
      "host" : "79c839e26e38",
      "ip" : "10.0.1.20",
      "version" : "1.7.5",
      "build" : "00f95f4",
      "http_address" : "inet[/10.0.1.20:9200]",
      "settings" : {
        "path" : {
          "logs" : "/usr/share/elasticsearch/logs",
          "home" : "/usr/share/elasticsearch",
          "repo" : "/usr/share/elasticsearch/snapshots"
        },
        "cluster" : {
          "name" : "elasticsearch"
        },
        "node" : {
          "name" : "es-node-1"
        },
        "discovery" : {
          "zen" : {
            "ping" : {
              "unicast" : {
                "hosts" : "es-node-2"
              }
            }
          }
        },
        "name" : "es-node-1",
        "client" : {
          "type" : "node"
        },
        "foreground" : "yes",
        "config" : {
          "ignore_system_properties" : "true"
        },
        "script" : {
          "inline" : "on",
          "update" : "on",
          "groovy" : "on"
        }
      }
    }
  }
}

[ec2-user@ip-10-5-3-235 ~]$ curl -XPUT 'http://localhost:9200/_snapshot/my_backup' -d '{
  "type": "fs",
  "settings": {
    "location": "/mnt/es-snaps",
    "compress": true
  }
}'
4

Hi David,
I checked the permissions of the mount point, still i get the same error as above.
Can you please help me out here?

Thanks in advance

5

Please format your code using </> icon as explained in this guide. It will make your post more readable.

Or use markdown style like:

```
CODE
```

I edited your post.

According to your Node settings, path.repo is set to /usr/share/elasticsearch/snapshots in all your nodes.

Did you restart? If you did, then you may be did not modify the right config file?

6

Hi David,

Thanks for your help.

When i ran this curl:
<curl -XPUT 'http://localhost:9200/_snapshot/ssg_backup' -d '{
"type": "fs",
"settings": {
"location": "/usr/share/elasticsearch/snapshots",
"compress": true
}
}'/>

I did not get any acknowledge message, i found the above error itself. But the repository was created when i checked it.
What may be the issue?

Thanks

7

Use this icon to format:

image
image.jpg900×108 20.8 KB

Can you share the full error log please?

8

Hi David,

Thanks for your help.
Sorry for texting it wrong way.

When i ran this curl:
Preformatted textcurl -XPUT 'http://localhost:9200/_snapshot/ssg_backup' -d '{
"type": "fs",
"settings": {
"location": "/usr/share/elasticsearch/snapshots",
"compress": true
}
}'Preformatted text

Preformatted text{"error":"RepositoryException[[my_backup] failed to create repository]; nested: CreationException[Guice creation errors:\n\n1) Error injecting constructor, org.elasticsearch.repositories.RepositoryException: [my_backup] location [/mnt/es-snaps] doesn't match any of the locations specified by path.repo\n at org.elasticsearch.repositories.fs.FsRepository.(Unknown Source)\n while locating org.elasticsearch.repositories.fs.FsRepository\n while locating org.elasticsearch.repositories.Repository\n\n1 error]; nested: RepositoryException[[my_backup] location [/mnt/es-snaps] doesn't match any of the locations specified by path.repo]; ","status":500}Preformatted text

I did not get any acknowledge message, i found the above error itself. But the repository was created when i checked it.
What may be the issue?

Thanks

9

Still wrong formatting.

image
image.jpg1306×324 94.3 KB

Please look at the preview window. It will shows you that it's not good before posting.
And edit your post if you see it's badly formatted.

Is the error you are seeing an old one? Or does it appear exactly when you send the PUT command?

I mean that may be there is no issue if the repo is created after all and that's only the old log you got before?

10 
`Thankyou David,

I ran the same curl to create new repo,

curl -XPUT 'http://localhost:9200/_snapshot/my_backup' -d '{

"type": "fs",
"settings": {
"location": "/usr/share/elasticsearch/snapshots",
"compress": true
}
}'

{"error":"RepositoryVerificationException[[my_backup] [3wU4GDPRSF-ki8XrvQf06g, 'RepositoryVerificationException[[my_backup] store location [/usr/share/elasticsearch/snapshots] is not accessible on the node [[es-node-2][3wU4GDPRSF-ki8XrvQf06g][8ae0627e0323][inet[/10.0.1.21:9300]]]]; nested: FileNotFoundException[/usr/share/elasticsearch/snapshots/tests-LLIqYYcIRumsCSOxhuFQ1Q-3wU4GDPRSF-ki8XrvQf06g (Read-only file system)]; '], [c2tNSJqSR6GeRgDj-U_8DA, 'RemoteTransportException[[es-node-1][inet[/10.0.1.20:9300]][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/usr/share/elasticsearch/snapshots] cannot be accessed on the node [[es-node-1][c2tNSJqSR6GeRgDj-U_8DA][6be39e1793be][inet[/10.0.1.20:9300]]]. This might indicate that the store [/usr/share/elasticsearch/snapshots] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node]; '], [4oS8jsaBSXiS2zo7hO14ng, 'RemoteTransportException[[es-node-3][inet[/10.0.1.22:9300]][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/usr/share/elasticsearch/snapshots] cannot be accessed on the node [[es-node-3][4oS8jsaBSXiS2zo7hO14ng][69171a5e5472][inet[/10.0.1.22:9300]]]. This might indicate that the store [/usr/share/elasticsearch/snapshots] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node]; ']]]","status":500}

But when i checked, my_backup has been created.

Thankyou. `

11

About formatting: OMG. Please, that's super annoying.

Just select the full lines you want to format, then click on </> button.
Or wrap them using

```
CODE
```

That's simple as that.

About the error, it means that /usr/share/elasticsearch/snapshots is not a shared drive between the 3 nodes.
It must be shared.

12 
Sorry again for the formatting part.

Since the elasticsearch is running in Docker container. 
/usr/share/elasticsearch/snapshots is mounted to /mnt/es-snaps (which is a NFS volume, shared with other two nodes.)
When i take a snapshot, the snapshot is taken to /mnt/es-snaps.
NFS is shareable but it is not working.
Do i have to make any change in the container level? Can you let me know what changes to make?

Thankyou

13

You don't have to format all your text but just the code.

Please...

Here elasticsearch is telling you that /usr/share/elasticsearch/snapshots is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node.

And I can also see Read-only file system. Which might give a clue here.

14

Hi David,

Since it is in the docker i need to check some more.
I have intiated snapshot, but i can see it on only 1 node (cluster of 3 nodes), Is it taking snapshot of Cluster or a single node?

Thank you

15

I have intiated snapshot, but i can see it on only 1 node (cluster of 3 nodes)

I'm unsure of what you are saying here.
It does not make sense to me. A snapshot is a backup of indices wherever the shards were allocated on.

A snapshot is not related to a node or to the number of nodes.

May be clarify what you are actually seeing?

16

I mean, if i want to take snapshot, do i have to take it on all 3 nodes? Or taking snapshot on 1 node is enough. Since it is a cluster.

17

Only one request is enough. The request is performed on the whole cluster

1 Like
18

Thankyou David.

I am checking with the permission issue on docker, will get back to you if i face any problem.

Thanks a lot for your help.

19

Hi David,

After completion of snapshot i see 282 shards following details (3 node cluster):

"    end_time_in_millis" : 1506176704543,
    "duration_in_millis" : 6892813,
    "failures" : [ ],
    "shards" : {
    "total" : 282,
    "failed" : 0,
    "successful" : 282
    }

But when i check on server i see 396 shards:
    curl -XGET 'localhost:9200/_stats?pretty'
    {
      "_shards" : {
        "total" : 396,
        "successful" : 396,
        "failed" : 0
      },

Has it taken the full backup?

20

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.