Not sure this is the right place, but i've migrated the elasticsearch portion of my existing ELK stack from individual EC2 nodes, onto AWS Elasticsearch Service.
My previous Elasticsearch cluster had automated backups with indices management to an s3 bucket, which im trying but failing to replicate, as I can't register the repo on the AWS Elasticsearch Service...
Due to the auth requirements on AWS, its unable to be done simply by curl and so im following the guide here:
however, the python "register" script example code in those instructions used to register the repo keeps failing.. First of all, i get errors
"TypeError: request() got an unexpected keyword argument 'json'"
unless i change the code from json=payload to data=payload... but after that change i get
401
{"Message":"Your request: '/_snapshot/elasticsearch-backups' is not allowed due to invalid input parameters."}
Not sure why the official AWS documentation is broken, nor can i find any info on the required parameters. Im thinking this documentation probably worked for es version 5 and something may have changed with elasticsearch 6 in terms of params required? nor can i find details for the es params for repo type s3 in the es version 6 docs.
Ive tried a couple other python scripts also, but they seem further out of date and i just get other code errors.
Any ideas or recommendations? surely many people have managed this successfully
Im running the latest versions for everything:
AWS Elasticsearch version 6.2
On my kibana node where im runnign the script; Python 2.7.x etc