Elastic 2.3.2 registering snapshot repo to Elastic File System mount failing

[impending_doom]

Hello,

quick setup specs:

2 client instances
3 master instances
3 data instances

S3 Snapshots work great.

EFS mounted to /backups on all instances using availability zone-specific IPs so that we also have a "local" copy.
Current permissions to the mount point:
755 elasticsearch:root

Using kopf to register snapshot repos, bash script to run snapshots.

All instances have the ability to read/write to/from EFS mount. When the snapshot repo creation is attempted, it is reflected on the mount. As in, the correct directories are created, etc. Eg.:

drwxr-xr-x 2 elasticsearch elasticsearch 4096 Jan 17 11:14 30day

The error I get upon hitting the 'Create' icon is:

[2017-01-17 11:14:54] Error executing request:
[2017-01-17 11:14:54] 
{"method":"POST","transformRequest":[null],"transformResponse":[null],"url":"http://client01.domain.com:9200/_snapshot/short_term","data":"{\"type\":\"fs\",\"settings\":{\"location\":\"30day\"}}","params":{},"headers":{"Accept":"application/json,
 text/plain, */*","Content-Type":"application/json;charset=utf-8"}}
  
[2017-01-17 11:14:54] REST API output:
[2017-01-17 11:14:54] 
{"error":{"root_cause":[{"type":"repository_verification_exception","reason":"[short_term]
 [88jdeo2NTbiRgSKAOG1-xQ, 
'RemoteTransportException[[node03][172.16.1.84:9300][internal:admin/repository/verify]];
 nested: RepositoryMissingException[[short_term] missing];'], 
[9c7rm-7ZSdGwqNLRXkvf7w, 
'RemoteTransportException[[node02][172.16.1.158:9300][internal:admin/repository/verify]];
 nested: RepositoryMissingException[[short_term] missing];'], 
[MHQfT5F-T0uhjeuqRmArFQ, 
'RemoteTransportException[[node01][172.16.1.160:9300][internal:admin/repository/verify]];
 nested: RepositoryMissingException[[short_term] 
missing];']]"}],"type":"repository_verification_exception","reason":"[short_term]
 [88jdeo2NTbiRgSKAOG1-xQ, 
'RemoteTransportException[[node03][172.16.1.84:9300][internal:admin/repository/verify]];
 nested: RepositoryMissingException[[short_term] missing];'], 
[9c7rm-7ZSdGwqNLRXkvf7w, 
'RemoteTransportException[[node02][172.16.1.158:9300][internal:admin/repository/verify]];
 nested: RepositoryMissingException[[short_term] missing];'], 
[MHQfT5F-T0uhjeuqRmArFQ, 
'RemoteTransportException[[node01][172.16.1.160:9300][internal:admin/repository/verify]];
 nested: RepositoryMissingException[[short_term] 
missing];']]"},"status":500}

Any light shed on this would be greatly appreciated.

Regards,

me

[dadoonet (David Pilato)]

Verb should be PUT not POST I think: https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html#_repositories