gce plugin with private access

vaboston · August 6, 2024, 4:57pm

Hello,
I'm using GCE discovery plugin on GCP VM, and i don't have nat gateway to join metadata.google.internal. I activated private access on my subnet and now i can use private.googleapis.com to ask metadata. But gce-disocvery send me :

Problem fetching instance list for zone europe-west1-c
java.net.SocketTimeoutException: Connect timed out

and tcpdump show me the outgoing request from plugin :

GET /computeMetadata/v1/instance/network-interfaces/0/ip HTTP/1.1
Accept-Encoding: gzip
User-Agent: Google-HTTP-Java-Client/1.44.1 (gzip)
metadata-flavor: Google
Host: metadata.google.internal

I tried to add in my /etc/hosts :

199.36.153.11 metadata.google.internal   //199.36.153.11 is IP for private.googleapis.com

but now i have 401 return code.
So, is there a way to use the plugin in private access ?
Thanks

dadoonet · August 6, 2024, 8:30pm

Welcome!

As I answered on slack, I'm wondering if there's a way with this

github.com

elastic/elasticsearch/blob/10b51179f97e493d38f3e81b71a429188df74ec9/plugins/discovery-gce/src/main/java/org/elasticsearch/plugin/discovery/gce/GceDiscoveryPlugin.java#L105


      
                      // Register GCE settings
                      GceInstancesService.PROJECT_SETTING,
                      GceInstancesService.ZONE_SETTING,
                      GceSeedHostsProvider.TAGS_SETTING,
                      GceInstancesService.REFRESH_SETTING,
                      GceInstancesService.RETRY_SETTING,
                      GceInstancesService.MAX_WAIT_SETTING
                  )
              );
          
              if (ALLOW_REROUTE_GCE_SETTINGS) {
                  settingList.add(GceMetadataService.GCE_HOST);
                  settingList.add(GceInstancesServiceImpl.GCE_ROOT_URL);
              }
              return Collections.unmodifiableList(settingList);
          }
          
          @Override
          public void close() throws IOException {
              IOUtils.close(gceInstancesService.get());
          }

And

github.com

elastic/elasticsearch/blob/10b51179f97e493d38f3e81b71a429188df74ec9/plugins/discovery-gce/src/main/java/org/elasticsearch/plugin/discovery/gce/GceDiscoveryPlugin.java#L46


      
          import java.util.Arrays;
          import java.util.Collections;
          import java.util.List;
          import java.util.Map;
          import java.util.function.Supplier;
          
          public class GceDiscoveryPlugin extends Plugin implements DiscoveryPlugin, Closeable {
          
              /** Determines whether settings those reroutes GCE call should be allowed (for testing purposes only). */
              private static final boolean ALLOW_REROUTE_GCE_SETTINGS = Booleans.parseBoolean(
                  System.getProperty("es.allow_reroute_gce_settings", "false")
              );
          
              public static final String GCE = "gce";
              protected final Settings settings;
              private static final Logger logger = LogManager.getLogger(GceDiscoveryPlugin.class);
              // stashed when created in order to properly close
              private final SetOnce<GceInstancesService> gceInstancesService = new SetOnce<>();
          
              static {
                  /*

It's supposed to be for tests only. But may be it would work...

vaboston · August 7, 2024, 8:44am

I didn't manage to pass the allow_reroute_gce_settings option but it doesn't matter, I'll do it differently, thank you for your time

Topic		Replies	Views
ECK- Discovery GCE plugin not working in GKE cluster Elastic Cloud on Kubernetes (ECK)	8	522	November 4, 2022
ES6.4 will not start up with gce discovery Elasticsearch	2	718	October 28, 2018
Elasticsearch 6.0.0 gce discovery failing Elasticsearch	13	1787	January 15, 2018
Why does discovery-gce need write permissions Elasticsearch	6	459	August 4, 2018
[ANN] Elasticsearch Google Compute Engine cloud plugin 2.5.0 released Elasticsearch	1	336	July 6, 2017

gce plugin with private access

Related topics