Silent failure to process AWS elb metrics

[mesiasc (camilo mesias)]

I have not managed to see any load balancer metrics.

I have a load balancer as part of an ECE deployment and can see the LB is working. Also it has metrics visible in the AWS console both for the LB itself and under cloudwatch.

I have metricbeat configured with a simple AWS config (having removed other working config temporarily) and have set log level to debug. Despite this the kibana dashboard for ELB is showing no data. The discovery page in kibana shows no records in metricbeat-* with aws namespace elb - only AWS/EC2.

I have checked the logs and there is scant output relating to elb specifically, and no errors to give me a clue as to the access.

Here's the config:

• module: aws
  period: 300s
  metricsets:
  • elb
    access_key_id: ''
    secret_access_key: ''

the credentials relate to a role with this policy:
"Action": [
"tag:GetResources",
"ec2:DescribeInstances",
"cloudwatch:GetMetricData",
"ec2:DescribeRegions",
"iam:ListAccountAliases",
"sts:GetCallerIdentity",
"cloudwatch:GetMetricStatistics",
"cloudwatch:ListMetrics"
],
"Resource": "*"

I think I've followed all the docs and have not managed to get any clues by debugging through the usual means. Please help!

[Kaiyan_Sheng]

@mesiasc Thanks for all the information! What kind of load balancer do you have in AWS? Right now the elb metricset only supports classic load balancer. I have a PR almost ready to add support for application elb and network elb: https://github.com/elastic/beats/pull/14123

At the same time, there is a workaround: use cloudwatch metricset instead. I can put a config example together for you if this is the problem!

[mesiasc (camilo mesias)]

Thanks for the reply, yes you are right we are not using the classic LB - we are using AWS::ElasticLoadBalancingV2::LoadBalancer (ALB)
I would be interested in your workaround if it would make the LB dashboard in Kibana work, if not then let's go for the PR

[Kaiyan_Sheng]

Thank you!!