Is it possible to connect Elasticsearch with App Engine with flexible environment via VPC Connector in Google Cloud Platform?

I have deployed my Django API in the App Engine with flexible environment. I have also created PostgreSQL instance in the Virtual Private Cloud and I connected my database with my Django App deployed in the App Engine using Google VPC Connector. Now I would like to do the same with the Elasticsearch. I enabled Elastic service via GCP portal and I was redirected to the Elastic website. During creation the deployment there is a security configuration. The problem is that when I try to create traffic filter I do not see the possibility to use Google VPC Connector. In all the documentations and tutorials I found information that it is only possibly when using AWS. Does anyone can confirm this information and if this is true maybe you can suggest any other solution as secure as possible? Of course I could create for example Compute Instance and configure everything on my own however automatic scaling and hourly pricing per GB are a great advantage from my perspective. Thank you in advance.

hi @white_rabbit

GCP Private Services Connect (Similar to AWS Private Link) is not available yet for Elastic Cloud, it is however on the roadmap as it is a highly requested feature. I do not have and ETA , keep your eyes open I am sure we will publicly announce when it is available.

You can use traffic filters to make sure only the traffic from your endpoints can access the elastic cloud deployments, but I understand that it's different than what you are looking for.

We do know if you deploy elastic cloud in the same region as your resources that are accessing it even though the elastic Cloud IP is public the traffic never leaves that region.

@stephenb Thank you for you quick response. It seems that the most reasonable solution at the moment would be to apply traffic filters. As you probably know Google App Engine doesn’t have a static IP because the service dynamically generates temporary IP addresses. Can you recommend the most optimal way to configure traffic filters while maintaining the highest possible security and data transfer between App Engine and Elasticsearch?

Well I am not an network engineer and security is a layered approach as I am sure you know and optimal is in the eye of the beholder....but seems to me you have 2 basic choices, with respect to traffic filters

  1. Setup the traffic filters to allow the full CIDR blocks for the regions your App Engine resources reside in. See here and here

  2. Or set up a proxy with a static IP to pass your App Engine traffic through and then setup the traffic filter to use that address.

For how to setup a proxy for Elastic Cloud see here