Hi Beats pros!
Is there any recommended way to pull and index AWS cost and usage report data into Elastic?
I've been using the built-in AWS Billing Metricbeat module to pull in billing data from AWS. From what I can tell, the only data that's available through this module is EstimatedCharges since that's the only metric sent through AWS CloudWatch under AWS/Billing
I'm mostly interested in BlendedCost data, i.e. the kind of info that's available via the aws CLI/API:
aws ce get-cost-and-usage --time-period Start=2020-06-19,End=2020-07-19 --granularity DAILY --metrics "BlendedCost" --group-by Type=DIMENSION,Key=SERVICE
Hi @stevewritescode 
I suggest to open an enhancement request in the Github repo directly https://github.com/elastic/beats/issues. Maybe that single metric can be added to the Metricset. Beats won't fetch all available metrics and it takes only a handful, waiting for some feedback from the community to add more. So I guess that your request will be welcome.
If you don't want to wait, there's no "fast" way to achieve this. One way would be to create a shell script to run the cli tool and output to a file. Then you setup a Ingest pipeline to parse the output data from AWS (using Grok, like many Filebeat modules) https://www.elastic.co/guide/en/elasticsearch/reference/master/ingest.html. Finally you setup filebeat to read that file and use the pipeline
Enhancement request added at https://github.com/elastic/beats/issues/20103. We can continue discussion there. Thanks for the quick reply!
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.