Is GCE Discovery still a thing?

Elastic Stack Elasticsearch
1

Hi there,

currently I'm investigating if it was sensible to enable the GCE-discovery in Elasticsearch.

According to this (older) thread, there are some hints about documentation and possibly implementation being outdated:

The documentation for the GCE-discovery in the current version 7.12: Setting up GCE Discovery | Elasticsearch Plugins and Integrations [7.12] | Elastic and I wonder if everything inside these docs is still up-to-date and if this was a sane thing to do.

Currently we're configuring ES with Ansible, passing initial cluster members with Ansible.
I wonder if GCE discovery would be of any help / improvement.

I currently would think the documentation is outdated and might be no longer valid.

2

It is indeed still a thing, but there's certainly a few areas of the docs that could do with updating. However ...

... do you mean the cluster.initial_master_nodes setting? That's nothing to do with discovery plugins like discovery-gce.

3

Well, I don't like the concept of cluster.initial_master_nodes that much, especially because I have to apply a static configuration to a cluster, that might be outdated later on.
I think this setting cluster.initial_master_nodes should even get removed after successful bootstrapping, like only stated on [this page](Important Elasticsearch configuration | Elasticsearch Guide [master] | Elastic):

After the cluster forms successfully for the first time, remove the cluster.initial_master_nodes setting from each nodes' configuration. Do not use this setting when restarting a cluster or adding a new node to an existing cluster.

Coming back to the original topic:

We're using GitHub - elastic/ansible-elasticsearch: Ansible playbook for Elasticsearch to configure Elasticsearch.

Unfortunately this role doesn't support plugins yet, so I have to either to extend our custom wrapper around this ansible-elasticsearch or extend the upstream-role to install the plugins.

4

Correct, these are the docs you're thinking of. But to repeat myself, this is nothing to do with discovery. For instance, you are required to set cluster.initial_master_nodes to the same value on every node, but there is no auto-discovery mechanism that can satisfy that requirement.

5

Thank you very much for the kind answers.
Yes, these topics are independent and I mixed them up.

GCE-discovery is working and I've to find out if tags work as I thought they do (like in the docs).
I will then also try to improve the documentation.

1 Like
6

@DavidTurner Do you know why the plugin does a call via this API compute.instances.updateNetworkInterface ?

I can see usage of these permissions:

compute.addresses.list
compute.instanceGroups.get
compute.instances.get
compute.instances.list
compute.instances.updateNetworkInterface
compute.zoneOperations.get
compute.zones.list

Only read-only access would be nice.

7

AFAICS and from what I recall, we are only calling the "get" APIs of the Google client. See elasticsearch/GceSeedHostsProvider.java at master · elastic/elasticsearch · GitHub

I was also surprised when I tested it that a compute-rw was needed by the Google client. :thinking:

8

No, sorry, I don't have a lot of knowledge about GCE specifics and I can't see anything obvious in the code. It gets info about the network interfaces if you use the _gce:..._ special values for network.host etc, but shouldn't be updating anything AFAIK

Can you forbid that specific permission? If so, does it break? Can you share logs and/or stack traces so we can see where it's being used?

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.