Using Metricbeat AWS Module on Private Subnet EC2

Elastic Stack Beats
1

Hi all!
My client wants to collect ec2, ebs, rds metrics using Metricbeat on private subnet ec2 instance with AWS module. So I searched for ways to do it.
Judging by document, there are common services to reach like IAM, STS, CloudWatch, EC2, Tagging right?
So I looked up for VPC Endpoints if there are services but IAM, Tagging wasn't there. I also tried using NAT Gateway and access through public internet but client won't be happy even though no one can access from public internet.

So here's the question: To use Metricbeat AWS module on private subnet ec2, is using public internet unavoidable?

Thanks in advance!

2

Hi @roh!

This is an interesting question, @Kaiyan_Sheng have we faced any similar case so far?

3

Hi,

Dose this issue resolved?
Looks we have similar issue.

Regards,
Jig

4

Hi, we have the same business requirements as @roh where our private subnet cannot be exposed to the internet.

Hoping for a prompt reply! Thank You!

5

Hello! Sorry for the late response. Right now in Metricbeat, AWS module collects monitoring metrics all from CloudWatch using AWS GetMetricData API. With your setup, do you know if you can make the Cloudwatch GetMetricData API call using AWS CLI on the private subnet EC2?

6

Hi, we know it uses AWS GetMetricData API and we also had all the necessary setup done such as:

  1. AWS PrivateLink Traffic Filtering to reach Elastic Cloud (Working for all the different beats)
  2. VPC endpoints created for EC2 metric set and tested calling the API to AWS :
  • ec2:DescribeInstances (Created & able to call API)
  • ec2:DescribeRegions (Created & able to call API)
  • cloudwatch:GetMetricData (Created & able to call API)
  • cloudwatch:ListMetrics (Created & able to call API)
  • sts:GetCallerIdentity (Created & able to call API)
  • iam:ListAccountAliases (Not available)

Therefore, we suspect if ListAccountAliases is the problem with this issue. Btw, ap-southeast-1 here.

7

@tanwk2 @Jignesh_Makwana I'm not sure this would help.
My client made Private Subnet EC2 to reach IAM endpoint somehow.
I asked him how and he told me if there's VPC that can reach IAM endpoint and Private EC2 is available to reach the VPC's EC2, it can reach it.
I'm not familiar with AWS and haven't tested it so I cannot guarantee.
I'll try to upload a result after the test.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.