# Fleet AWS Billing integration

**URL:** <https://discuss.elastic.co/t/fleet-aws-billing-integration/311138>\
**Category:** Beats\
**Tags:** fleet, elastic-agent, integrations\
**Created:** [August 1, 2022, 3:41pm UTC](https://discuss.elastic.co/t/fleet-aws-billing-integration/311138 "2022-08-01T15:41:23Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![aricau](https://avatars.discourse-cdn.com/v4/letter/a/4491bb/32.png) [@aricau](https://discuss.elastic.co/u/aricau)\
**Post date:** [August 1, 2022, 3:41pm UTC](https://discuss.elastic.co/t/fleet-aws-billing-integration/311138/1 "2022-08-01T15:41:23Z")

</div>

Hello all,

I'm trying to collect billing data with the AWS Billing integration using the elastic agent managed by Fleet. Running version 8.3.3

The agent appears to be running fine, with valid secret access key and permissions, but no data gets ingested.  
The logs from the agent show the following error:

```auto
16:27:44.333
elastic_agent.metricbeat
[elastic_agent.metricbeat][error] costexplorer GetCostAndUsageRequest failed: SerializationError: failed decoding JSON RPC error response
caused by: invalid character '<' looking for beginning of value

```

Running out of ideas. Should I try with metricbeat perhaps?  
Any help would be greatly appreciated.

---

<div class="post-metadata">

**Author:** ![TiagoQueiroz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tiagoqueiroz/32/107061_2.png) [@TiagoQueiroz](https://discuss.elastic.co/u/TiagoQueiroz)\
**Post date:** [August 2, 2022, 2:41pm UTC](https://discuss.elastic.co/t/fleet-aws-billing-integration/311138/2 "2022-08-02T14:41:49Z")

</div>

@Andrea_Spacca could you shed some light here? It seems to be an issue with the AWS integration.

---

<div class="post-metadata">

**Author:** ![aricau](https://avatars.discourse-cdn.com/v4/letter/a/4491bb/32.png) [@aricau](https://discuss.elastic.co/u/aricau)\
**Post date:** [August 3, 2022, 3:04pm UTC](https://discuss.elastic.co/t/fleet-aws-billing-integration/311138/3 "2022-08-03T15:04:28Z")

</div>

Tried again using metricbeat version 7.17.0 with the following configuration:

```auto
logging.level: info
logging.selectors: ["*"]

metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.kibana:
  host: "http://kibana:5601"
  username: "elastic"
  password: "<PASSWORD>"

processors:
  - add_cloud_metadata: ~
  - add_docker_metadata: ~

output.elasticsearch:
  hosts: ["http://elasticsearch:9200"]
  username: "elastic"
  password: "<PASSWORD>"

metricbeat.modules:
- module: aws
  period: 60s
  access_key_id: <KEY ID>
  secret_access_key: <SECRET ACCESS KEY>
  metricsets:
    - billing
  cost_explorer_config:
    group_by_dimension_keys:
      - "AZ"
      - "INSTANCE_TYPE"
      - "SERVICE"
      - "LINKED_ACCOUNT"
    group_by_tag_keys:
      - "aws:createdBy"

```

and I'm getting an error message:

```auto
ERROR [aws.billing] billing/billing.go:254 costexplorer GetCostAndUsageRequest failed: ValidationException: Start date (and hour) should be before end date (and hour)
        status code: 400, request id: 5aa02481-0576-409c-8eaf-809c4657e383

```

Same error message running version 8.3.3

---

<div class="post-metadata">

**Author:** ![aricau](https://avatars.discourse-cdn.com/v4/letter/a/4491bb/32.png) [@aricau](https://discuss.elastic.co/u/aricau)\
**Post date:** [August 3, 2022, 3:34pm UTC](https://discuss.elastic.co/t/fleet-aws-billing-integration/311138/4 "2022-08-03T15:34:53Z")

</div>

Confirmed. I get the same with the elastic agent version 8.3.3.

```auto
[elastic_agent.metricbeat][error] aws GetMetricDataResults failed with error GetMetricData with Paginator: ValidationError: The parameter StartTime must not equal parameter EndTime.

	status code: 400, request id: 43614125-ec4e-46ad-8b21-6b282bc4dac7, skipping region us-east-1

[elastic_agent.metricbeat][error] costexplorer GetCostAndUsageRequest failed: SerializationError: failed decoding JSON RPC error response
caused by: invalid character '<' looking for beginning of value

```

---

<div class="post-metadata">

**Author:** ![Andrea\_Spacca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrea_spacca/32/76976_2.png) [@Andrea\_Spacca](https://discuss.elastic.co/u/Andrea_Spacca)\
**Post date:** [August 8, 2022, 1:43am UTC](https://discuss.elastic.co/t/fleet-aws-billing-integration/311138/5 "2022-08-08T01:43:48Z")

</div>

hello @aricau

sorry for the late feedback, we'll look into that

what version of the AWS integration package are you using?

---

<div class="post-metadata">

**Author:** ![Andrea\_Spacca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrea_spacca/32/76976_2.png) [@Andrea\_Spacca](https://discuss.elastic.co/u/Andrea_Spacca)\
**Post date:** [August 8, 2022, 2:00am UTC](https://discuss.elastic.co/t/fleet-aws-billing-integration/311138/6 "2022-08-08T02:00:56Z")

</div>

hello @aricau

it seems the problem reside in the `period` setting.  
did you use a custom value by chance?

---

<div class="post-metadata">

**Author:** ![aricau](https://avatars.discourse-cdn.com/v4/letter/a/4491bb/32.png) [@aricau](https://discuss.elastic.co/u/aricau)\
**Post date:** [August 8, 2022, 8:48am UTC](https://discuss.elastic.co/t/fleet-aws-billing-integration/311138/7 "2022-08-08T08:48:13Z")

</div>

Hi @Andrea_Spacca . I'm using AWS Billing version 1.18.2  
I did set my `period` to 2 hours. Is that too low?

---

<div class="post-metadata">

**Author:** ![Andrea\_Spacca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrea_spacca/32/76976_2.png) [@Andrea\_Spacca](https://discuss.elastic.co/u/Andrea_Spacca)\
**Post date:** [August 8, 2022, 10:02am UTC](https://discuss.elastic.co/t/fleet-aws-billing-integration/311138/8 "2022-08-08T10:02:35Z")

</div>

hi @aricau

there is bug that provokes that error if the `period` is less than `24h`

as a mitigation you can set that value, until the bug will be fixed

---

<div class="post-metadata">

**Author:** ![aricau](https://avatars.discourse-cdn.com/v4/letter/a/4491bb/32.png) [@aricau](https://discuss.elastic.co/u/aricau)\
**Post date:** [August 8, 2022, 11:22am UTC](https://discuss.elastic.co/t/fleet-aws-billing-integration/311138/9 "2022-08-08T11:22:54Z")

</div>

Had a look at the source code over in [beats/billing.go at main · elastic/beats · GitHub](https://github.com/elastic/beats/blob/main/x-pack/metricbeat/module/aws/billing/billing.go) and realised that the module is requesting daily billing data and the start & end date is calculated based on the period.

Can confirm it's now working if `period` is set to `24h`.

Thanks.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 5, 2022, 11:23am UTC](https://discuss.elastic.co/t/fleet-aws-billing-integration/311138/10 "2022-09-05T11:23:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.