RDS module only getting metrics of one instance

[miguel.d]

Hello!

We tried the rds module with basic settings

- module: aws
  period: 60s
  metricsets:
    - rds
  access_key_id: 'accesskey'
  secret_access_key: 'secretkey'

It is able to pull all the db_instance properly but all of the other fields return the same, which is true for one of the instances but not all of it.

Here's an example index output from ES:

  "rds": {

    "free_storage.bytes": 18326547660.8,

    "read_io.ops_per_sec": 0.23332555581480618,

    "maximum_used_transaction_ids": 45875,

    "db_instance": {

      "arn": "arn:aws:rds:ca-central-1:xxxxx:db:sampledb1",

      "class": "db.t2.micro",

      "identifier": "sampledb1",

      "status": "available"

    },

    "transaction_logs_generation": 223696.2350282486,

    "disk_queue_depth": 0.06960639691711205,

    "disk_usage": {

      "bin_log.bytes": 5874.8,

      "replication_slot.mb": 4096,

      "transaction_logs.mb": 2348818769

    }

   



   

  "rds": {

    "free_storage.bytes": 18326547660.8,

    "read_io.ops_per_sec": 0.23332555581480618,

    "maximum_used_transaction_ids": 45875,

    "db_instance": {

      "arn": "arn:aws:rds:ca-central-1:xxxxx:db:sampledb2",

      "class": "db.t2.micro",

      "identifier": "sampledb2",

      "status": "available"

    },

    "transaction_logs_generation": 223696.2350282486,

    "disk_queue_depth": 0.06960639691711205,

    "disk_usage": {

      "bin_log.bytes": 5874.8,

      "replication_slot.mb": 4096,

      "transaction_logs.mb": 2348818769

    }

   



   

  "rds": {

    "free_storage.bytes": 18326547660.8,

    "read_io.ops_per_sec": 0.23332555581480618,

    "maximum_used_transaction_ids": 45875,

    "db_instance": {

      "arn": "arn:aws:rds:ca-central-1:xxxxx:db:sampledb3",

      "class": "db.t2.micro",

      "identifier": "sampledb3",

      "status": "available"

    },

    "transaction_logs_generation": 223696.2350282486,

    "disk_queue_depth": 0.06960639691711205,

    "disk_usage": {

      "bin_log.bytes": 5874.8,

      "replication_slot.mb": 4096,

      "transaction_logs.mb": 2348818769

    }

[Kaiyan_Sheng]

Hi @miguel.d, what version of metricbeat are you running? Let me try to reproduce it on my side and see what we can do to fix this. Thanks!

[miguel.d]

Hi @Kaiyan_Sheng, we are using 7.4.

Thanks!

[Kaiyan_Sheng]

@miguel.d Thank you! I was able to reproduce it and it is a bug in my code I'm working on a PR to fix this: https://github.com/elastic/beats/pull/14702

Mean time, if you want to give cloudwatch metricset a try with namespace AWS/RDS, that should collect all the rds metrics for you as a workaround. For example:

- module: aws
  period: 300s
  metricsets:
    - cloudwatch
  metrics:
    - namespace: AWS/RDS
  access_key_id: 'accesskey'
  secret_access_key: 'secretkey'

[miguel.d]

@Kaiyan_Sheng That's exactly what we use as a workaround!

Looking forward for the fix!

Thanks!