Snaphsot repository Read Only URL via FTP

I have an Elasticsearch cluster of 3 nodes with version 7.15.0.
I want to add a read-only URL repository soi can fetch a snapshot for restore.

PUT /_snapshot/ftp_snaphoot
{
  "type": "url",
  "settings": {
    "url": "ftp://username:password@ftp_server_ip_adress"
  }
}

I got this error

{
  "error" : {
    "root_cause" : [
      {
        "type" : "repository_exception",
        "reason" : "[ftp_snaphoot] Could not determine repository generation from root blobs"
      }
    ],
    "type" : "repository_exception",
    "reason" : "[ftp_snaphoot] Could not determine repository generation from root blobs",
    "caused_by" : {
      "type" : "security_exception",
      "reason" : "access denied (\"java.net.SocketPermission\" \"localhost:0\" \"listen,resolve\")"
    }
  },
  "status" : 500
}

When the username login to ftp_server_ip_adress using FTP, it's directly located into a respository with a snapshot. This respository is used as FS to take snapshot from the cluster.

This is the stack trace reported by in Elasticsearch logs

{"type": "server", "timestamp": "2021-10-23T16:33:46,248Z", "level": "WARN", "component": "r.suppressed", "cluster.name": "docker-cluster", "node.name": "elasticsearch_1", "message": "path: /_snapshot/ftp_snaphoot/_verify, params: {repository=ftp_snaphoot}", "cluster.uuid": "JLG6niOySXmX9mJkzv1I0w", "node.id": "D0H2EJblRmaBrwohfQBFPw" ,
"stacktrace": ["org.elasticsearch.transport.RemoteTransportException: [elasticsearch_3][172.18.0.3:9300][cluster:admin/repository/verify]",
"Caused by: org.elasticsearch.repositories.RepositoryVerificationException: [ftp_snaphoot] path  is not accessible on master node",
"Caused by: java.lang.SecurityException: access denied (\"java.net.SocketPermission\" \"localhost:0\" \"listen,resolve\")",
"at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]",
"at java.security.AccessController.checkPermission(AccessController.java:1036) ~[?:?]",
"at java.lang.SecurityManager.checkPermission(SecurityManager.java:408) ~[?:?]",
"at java.lang.SecurityManager.checkListen(SecurityManager.java:990) ~[?:?]",
"at java.net.Socket.bind(Socket.java:691) ~[?:?]",
"at sun.net.ftp.impl.FtpClient.openPassiveDataConnection(FtpClient.java:691) ~[?:?]",
"at sun.net.ftp.impl.FtpClient.openDataConnection(FtpClient.java:806) ~[?:?]",
"at sun.net.ftp.impl.FtpClient.getFileStream(FtpClient.java:1384) ~[?:?]",
"at sun.net.www.protocol.ftp.FtpURLConnection.getInputStream(FtpURLConnection.java:460) ~[?:?]",
"at java.net.URL.openStream(URL.java:1192) ~[?:?]",
"at java.security.AccessController.doPrivileged(AccessController.java:554) ~[?:?]",
"at org.elasticsearch.common.blobstore.url.URLBlobContainer.getInputStream(URLBlobContainer.java:142) ~[?:?]",
"at org.elasticsearch.common.blobstore.url.URLBlobContainer.readBlob(URLBlobContainer.java:110) ~[?:?]",
"at org.elasticsearch.repositories.blobstore.BlobStoreRepository.readSnapshotIndexLatestBlob(BlobStoreRepository.java:2563) ~[elasticsearch-7.15.0.jar:7.15.0]",
"at org.elasticsearch.repositories.blobstore.BlobStoreRepository.latestIndexBlobId(BlobStoreRepository.java:2554) ~[elasticsearch-7.15.0.jar:7.15.0]",
"at org.elasticsearch.repositories.blobstore.BlobStoreRepository.startVerification(BlobStoreRepository.java:1655) ~[elasticsearch-7.15.0.jar:7.15.0]",
"at org.elasticsearch.repositories.RepositoriesService$4.doRun(RepositoriesService.java:392) ~[elasticsearch-7.15.0.jar:7.15.0]",
"at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:737) ~[elasticsearch-7.15.0.jar:7.15.0]",
"at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-7.15.0.jar:7.15.0]",
"at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) ~[?:?]",
"at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) ~[?:?]",
"at java.lang.Thread.run(Thread.java:831) [?:?]"] }

It looks like FTP-based URL repositories simply don't work: the FTP client is trying to do something that the security manager forbids. This code hasn't changed in well over a decade so I guess you're the first person to actually try doing this. Please report it on Github as a bug - at the very least we should remove ftp from the list of URL schemes that are documented as supported. In the meantime I'd suggest using HTTP instead.

Thanks @DavidTurner for your feedbakc
I reported an issue here Snaphsot repository Read Only URL via FTP not accepting FTP URL · Issue #79685 · elastic/elasticsearch · GitHub
Hope that will help.

My idea is to use FTP repo on a remote cluster when i need to restore the snaphot from the main cluster. and using an FTP repo (ReadOnly) is a good option for me to avoid mounting NFS ...

1 Like

Yes, makes sense, but FTP is a fairly obscure protocol these days as you have discovered. It doesn't do anything that HTTP(S) can't.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.