I have a local Elasticsearch cluster consisting of 3 master nodes and 3 data nodes, all the 6 servers are VMs inside Google Cloud. I create a snapshot repository to google GCS bucket, and save snapshots there.
1, only GCP service account's key.json file works for authentication to GCS repository plugin. I can not fall back to use the GCP service account associated with the GCP Virtual Machines. While the link says it can fall back. Getting started | Elasticsearch Plugins and Integrations [7.16] | Elastic
The running Elasticsearch version is 7.8.0.
2, The GCP service account key.json has to be installed into keystore file in all 6 VMs, to get repository successfully set up, why? I installed it onto only one master node, then try repository creation but it failed with 500 return code.
Anyone in this area could shed a light into it, I'm trying to avoid key.json file, as it is rotated monthly for security reasons. Thanks,
What did you try (i.e. what exactly was in your config) and what happened to indicate that it didn't work? I believe as long as the credentials_file setting is absent then this should work.
Note that this is a secure setting so you will need to use elasticsearch-keystore list to ensure that it is removed.
All nodes need direct access to the repository to avoid the dreadful bottleneck (and massive network traffic costs) that would result from needing to send all the data through one privileged node.
This version is past EOL which severely limits our ability to investigate problems. You should upgrade to a supported version ASAP.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.