Hi community, and thanks for all the help you've given everyone for years and years.
Today, I'm facing a very interesting situation.
I'm finally going around on upgrading a quite old Elasticsearch 5.0 cluster hosted in raw AWS EC2 instances using the Elastic provided Docker image.
We want to move it to the cutting edge 8.4.
As far as I know, I have to thread slowly, first moving from 5.0 to 6.8, then 6.8 to 7.17 and finally from 7.17 to 8.4.
So, I created a S3 repo on the 5.0 cluster, as such:
$ curl -XPOST "http://localhost:9200/_snapshot/s3_snapshots_5_0" -d '{
"type": "s3",
"settings": {
"bucket": "backup-lt",
"base_path": "/snapshots/es/lt-5-0",
"max_restore_bytes_per_sec": "20mb",
"compress": "true",
"max_snapshot_bytes_per_sec": "20mb"
}
}'
{"acknowledged":true}
From there I ran a couple successful snapshots:
$ curl -XGET "http://localhost:9200/_snapshot/s3_snapshots_5_0/_all" | jq
{
"snapshots": [
{
"snapshot": "initial",
"uuid": "FtDkESulSr-ZQY0mX3_7iw",
"version_id": 5000099,
"version": "5.0.0",
"indices": [
".tasks",
"basics-cp"
],
"state": "SUCCESS",
"start_time": "2022-10-03T17:52:51.953Z",
"start_time_in_millis": 1664819571953,
"end_time": "2022-10-03T18:33:23.237Z",
"end_time_in_millis": 1664822003237,
"duration_in_millis": 2431284,
"failures": [],
"shards": {
"total": 11,
"failed": 0,
"successful": 11
}
},
{
"snapshot": "20221004",
"uuid": "gO_QPXylS8Kous-Lr1oyXg",
"version_id": 5000099,
"version": "5.0.0",
"indices": [
".tasks",
"basics-cp"
],
"state": "SUCCESS",
"start_time": "2022-10-04T19:10:30.909Z",
"start_time_in_millis": 1664910630909,
"end_time": "2022-10-04T19:10:32.311Z",
"end_time_in_millis": 1664910632311,
"duration_in_millis": 1402,
"failures": [],
"shards": {
"total": 11,
"failed": 0,
"successful": 11
}
}
]
}
Then, in the separate 6.8 cluster, I do pretty much the same (except for it being readonly):
$ curl -XPOST "http://localhost:9200/_snapshot/s3_snapshots_5_0" -H "Content-Type:application/json" -d '{
"type": "s3",
"settings": {
"bucket": "backup-lt",
"base_path": "/snapshots/es/lt-5-0",
"max_restore_bytes_per_sec": "20mb",
"compress": "true",
"max_snapshot_bytes_per_sec": "20mb",
"readonly": "true"
}
}'
{"acknowledged":true}
However, I cannot retrieve any snapshots from it:
$ curl -XGET "http://localhost:9200/_snapshot/s3_snapshots_5_0/_all"
{"snapshots":[]}
Again, both of the clusters are being hosted in AWS as raw EC2 instances using the base Elastic provided Docker image (with minor changes to add the repository-s3 and discovery-ec2 plugins), and both have the exact same role permissions (including read and write permissions to S3).
I have checked S3 access from the instances using AWS CLI, and it is working as intended. Checked the Docker logs for the Elasticsearch container and no errors appear.
Am I missing something that changed between 5.0 and 6.8? Or is it something else?
Thanks for your help!
EDIT #1: using jq to reformat the json requests and responses
)