Elastic Cloud Private Link connectivity for clients with CloudID

[kiwibel (Vadim Chausov)]

Hi,

We are using Elastic Cloud (AWS) and the ES client on the app side is configured with CloudID.
We have added Private Link to access our deployment as described here:
https://www.elastic.co/guide/en/cloud/current/ec-traffic-filtering-vpc.html

The connectivity to EC is working fine from the VPC with new URL (https://XXX.vpce.ap-southeast-2.aws.elastic-cloud.com:9243).
However, as the client is using CloudID there seems to be a problem. I suspect it default to the publicly accessible URL (https://XXX.ap-southeast-2.aws.found.io:9243).

Is there any way to get Private Link working with a Node client using Cloud ID?
Here's a sample of the client setup:

    const { Client } = require('@elastic/elasticsearch')
    const client = new Client({
      cloud: {
        id: 'name:bG9jYWxob3N0JGFiY2QkZWZnaA==',
      },
      auth: {
        username: 'elastic',
        password: 'changeme'
      }
    })

Cheers,
Vadim

[stephenb (Stephen Brown)]

Hi @kiwibel

Welcome to the community and thank you for using Elastic Cloud.

Yes the cloud.id decodes to the normal public URL.

Once the VPC PrivateLink us set up unless you specifically also allow the public IP ingress it will no longer be available.

The Node / JavaScript client (assuming it is in your AWS VPC) can simply connect with basic authentication using the new URL.

const { Client } = require('@elastic/elasticsearch')
const client = new Client({
  node: 'https://XXX.vpce.ap-southeast-2.aws.elastic-cloud.com:9243',
  auth: {
    username: 'elastic',
    password: 'changeme'
  }
})

OR you can also add back / allow Public IPs in the traffic filter 0.0.0.0/0 which is perhaps a bit counter to setting up the PrivateLink in the first place.

Then you could use the cloud.id

Hope that helps

[kiwibel (Vadim Chausov)]

Thanks @stephenb

We used to have URL in client's config the way you described but switched to Cloud ID as it has some benefits. I was wondering if cloud.id could somehow be aware of PrivateLink for the deployment and resolve to a private IP dynamically.

It seems we will need to revert to using URL.

Thanks,
Vadim

[stephenb (Stephen Brown)]

The cloud.id decodes to the public URL / Endpoint that is created when you create the deployment. I don't think it's possible for it to know ahead of time the new DNS VPC endpoint URL that's created by you the user when you create private link afterwards.

Refining / thinking a little bit further....You should be able to add IP traffic filters on the Elastic Cloud deployment to include the source ranges for your clients. Then you should be able to use the cloud.id from the clients.

That traffic would still go to the original public IP address the sources could be filtered.

Hope that makes sense

[kiwibel (Vadim Chausov)]

Yep. Totally makes sense if we still need public access to Elastic Cloud.
That however defeat the purpose for using PrivateLink as you mentioned before.

Thanks again for the super prompt reply. Happy to join Elastic community

Cheers,
Vadim