Metricbeat AWS Billing Module does not combine Group By options

Elastic Stack Beats
1

It looks like the Metricbeat AWS Billing module takes in an arbitrary number of Cost Explorer Group by Dimensions, and then iterates over them individually to generate documents.

Ideally it should allow you to pair up dimension keys, as you can send 2 at a time to the API. The best example of this, is if you're using a consolidated billing account.

In the current config, you can pull the "SERVICE" and "LINKED_ACCOUNT" Dimensions. Which gives you two groups of documents.

  1. How much you spent per service across all accounts.
  2. The total cost of each account.

This means it's impossible to drill down and see a breakdown of what happened in each account.

The AWS API does allow two dimensions in one request. In which case metricbeat would receive every unique pair of Account and Service type. Which would be a much more usefull data set.

2

Taking a second look, it appears that the module is taking 1 group by dimension and 1 group by tag and therefore maxing out the 2 it can have.

Ideally the config should take in pairs(or singles) of group_by options. Then run against each pair to generate documents.

3

Hi @Thomas_Cate Welcome to the community!

Good insight, perhaps you would consider opening a feature request in the beats repo here

4

Sounds good, I opened up a feature request in GH.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.