Good day,
I'm currently trying to collect billing information from AWS by using the billing metricset of the Metricbeat AWS module, but the received data does not contain any billing information.
my metricbeat.yml:
metricbeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
setup.template.settings:
index.number_of_shards: 1
index.codec: best_compression
setup.kibana:
host: "localhost:5601"
output.elasticsearch:
hosts: ["localhost:9200"]
protocol: "https"
username: elastic
password: <pw>
ssl.certificate_authorities: ["<path-to>/ca.crt"]
processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
The aws module ist enabled. My /etc/metricbeat/modules.d/aws.yml:
- module: aws
period: 24h
metricsets:
- billing
cost_explorer_config:
group_by_dimension_keys:
- "AZ"
- "INSTANCE_TYPE"
- "SERVICE"
group_by_tag_keys:
- "aws:createdBy"
access_key_id: '<access-key>'
secret_access_key: '<secret-access-key>'
I expected to receive data that contains fields like:
{
"aws": {
"billing": {
"AmortizedCost": {...},
"BlendedCost": {...}},
"event": {
"dataset": "aws.billing",..
},
"metricset": {
"name": "billing",
}}
But instead my I don't receive any billing information. My event.dataset and metricsets look like this:
"event": {
"dataset": "system.process",
"dataset": "system.network",
"dataset": "system.memory",
"dataset": "system.cpu",
"dataset": "system.load",
},
"metricset": {
"name": "system.process",
"name": "system.network",
"name": "system.memory",
"name": "system.cpu",
"name": "system.load",
},
My AWS Admin assures me, that I have the necessary rights:
cloudwatch:GetMetricData
cloudwatch:ListMetrics
tag:getResources
sts:GetCallerIdentity
iam:ListAccountAliases
ce:GetCostAndUsage
I receive the same data, when I use the ec2 metricset or the cloudwatch metricset. It doesn't seem to make any difference which metricset I set up.
Edit: I also get only data about one instance, although there are three instances deployed on aws.
I am at my wit's end and would appreciate any help.
